2023-02-09 16:54:01,423 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-09 16:54:01,693 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4 2023-02-09 16:54:01,707 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 16:54:01,708 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6, deleteOnExit=true 2023-02-09 16:54:01,708 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 16:54:01,708 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/test.cache.data in system properties and HBase conf 2023-02-09 16:54:01,709 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 16:54:01,709 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/hadoop.log.dir in system properties and HBase conf 2023-02-09 16:54:01,710 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 16:54:01,710 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 16:54:01,710 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 16:54:01,820 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 16:54:02,217 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 16:54:02,221 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 16:54:02,222 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 16:54:02,222 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 16:54:02,222 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 16:54:02,223 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 16:54:02,223 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 16:54:02,223 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 16:54:02,223 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 16:54:02,224 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 16:54:02,224 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/nfs.dump.dir in system properties and HBase conf 2023-02-09 16:54:02,225 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/java.io.tmpdir in system properties and HBase conf 2023-02-09 16:54:02,225 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 16:54:02,225 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 16:54:02,225 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 16:54:02,741 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 16:54:02,746 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 16:54:03,042 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 16:54:03,222 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 16:54:03,237 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 16:54:03,275 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 16:54:03,308 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/java.io.tmpdir/Jetty_localhost_40979_hdfs____.hgz9tj/webapp 2023-02-09 16:54:03,458 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40979 2023-02-09 16:54:03,469 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 16:54:03,469 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 16:54:03,919 WARN [Listener at localhost/44699] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 16:54:03,991 WARN [Listener at localhost/44699] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 16:54:04,009 WARN [Listener at localhost/44699] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 16:54:04,015 INFO [Listener at localhost/44699] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 16:54:04,019 INFO [Listener at localhost/44699] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/java.io.tmpdir/Jetty_localhost_33915_datanode____f4so4k/webapp 2023-02-09 16:54:04,115 INFO [Listener at localhost/44699] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33915 2023-02-09 16:54:04,450 WARN [Listener at localhost/41187] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 16:54:04,469 WARN [Listener at localhost/41187] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 16:54:04,476 WARN [Listener at localhost/41187] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 16:54:04,477 INFO [Listener at localhost/41187] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 16:54:04,486 INFO [Listener at localhost/41187] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/java.io.tmpdir/Jetty_localhost_37145_datanode____.u35eob/webapp 2023-02-09 16:54:04,606 INFO [Listener at localhost/41187] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37145 2023-02-09 16:54:04,615 WARN [Listener at localhost/37815] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 16:54:04,633 WARN [Listener at localhost/37815] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 16:54:04,636 WARN [Listener at localhost/37815] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 16:54:04,637 INFO [Listener at localhost/37815] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 16:54:04,642 INFO [Listener at localhost/37815] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/java.io.tmpdir/Jetty_localhost_41363_datanode____erpg56/webapp 2023-02-09 16:54:04,805 INFO [Listener at localhost/37815] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41363 2023-02-09 16:54:04,814 WARN [Listener at localhost/39015] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 16:54:05,609 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc30b2e829b63332e: Processing first storage report for DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551 from datanode 627681a5-99ed-41c7-b973-a8e3d411bd34 2023-02-09 16:54:05,610 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc30b2e829b63332e: from storage DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551 node DatanodeRegistration(127.0.0.1:40493, datanodeUuid=627681a5-99ed-41c7-b973-a8e3d411bd34, infoPort=42199, infoSecurePort=0, ipcPort=41187, storageInfo=lv=-57;cid=testClusterID;nsid=463676963;c=1675961642820), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcd5ec62ce1b53aab: Processing first storage report for DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2 from datanode 762ffd31-8114-414e-bb4b-20a4d577f100 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcd5ec62ce1b53aab: from storage DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2 node DatanodeRegistration(127.0.0.1:40921, datanodeUuid=762ffd31-8114-414e-bb4b-20a4d577f100, infoPort=37285, infoSecurePort=0, ipcPort=39015, storageInfo=lv=-57;cid=testClusterID;nsid=463676963;c=1675961642820), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x99606ab43750b29f: Processing first storage report for DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b from datanode e1d56305-25f8-42c9-ab95-be1e251b8441 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x99606ab43750b29f: from storage DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b node DatanodeRegistration(127.0.0.1:42345, datanodeUuid=e1d56305-25f8-42c9-ab95-be1e251b8441, infoPort=36091, infoSecurePort=0, ipcPort=37815, storageInfo=lv=-57;cid=testClusterID;nsid=463676963;c=1675961642820), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc30b2e829b63332e: Processing first storage report for DS-1397f4fe-4f29-4a91-871e-4b080ff28c72 from datanode 627681a5-99ed-41c7-b973-a8e3d411bd34 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc30b2e829b63332e: from storage DS-1397f4fe-4f29-4a91-871e-4b080ff28c72 node DatanodeRegistration(127.0.0.1:40493, datanodeUuid=627681a5-99ed-41c7-b973-a8e3d411bd34, infoPort=42199, infoSecurePort=0, ipcPort=41187, storageInfo=lv=-57;cid=testClusterID;nsid=463676963;c=1675961642820), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcd5ec62ce1b53aab: Processing first storage report for DS-267992a2-2c9d-41ac-b016-bdba4653b4f4 from datanode 762ffd31-8114-414e-bb4b-20a4d577f100 2023-02-09 16:54:05,611 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcd5ec62ce1b53aab: from storage DS-267992a2-2c9d-41ac-b016-bdba4653b4f4 node DatanodeRegistration(127.0.0.1:40921, datanodeUuid=762ffd31-8114-414e-bb4b-20a4d577f100, infoPort=37285, infoSecurePort=0, ipcPort=39015, storageInfo=lv=-57;cid=testClusterID;nsid=463676963;c=1675961642820), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 16:54:05,612 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x99606ab43750b29f: Processing first storage report for DS-3d672513-e429-4784-b47b-bd8fcb2a2f91 from datanode e1d56305-25f8-42c9-ab95-be1e251b8441 2023-02-09 16:54:05,612 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x99606ab43750b29f: from storage DS-3d672513-e429-4784-b47b-bd8fcb2a2f91 node DatanodeRegistration(127.0.0.1:42345, datanodeUuid=e1d56305-25f8-42c9-ab95-be1e251b8441, infoPort=36091, infoSecurePort=0, ipcPort=37815, storageInfo=lv=-57;cid=testClusterID;nsid=463676963;c=1675961642820), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 16:54:05,639 DEBUG [Listener at localhost/39015] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4 2023-02-09 16:54:05,703 INFO [Listener at localhost/39015] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/zookeeper_0, clientPort=59211, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 16:54:05,719 INFO [Listener at localhost/39015] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=59211 2023-02-09 16:54:05,730 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:05,733 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:06,399 INFO [Listener at localhost/39015] util.FSUtils(479): Created version file at hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9 with version=8 2023-02-09 16:54:06,399 INFO [Listener at localhost/39015] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/hbase-staging 2023-02-09 16:54:06,704 INFO [Listener at localhost/39015] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 16:54:07,175 INFO [Listener at localhost/39015] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 16:54:07,209 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,210 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,210 INFO [Listener at localhost/39015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 16:54:07,210 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,210 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 16:54:07,349 INFO [Listener at localhost/39015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 16:54:07,421 DEBUG [Listener at localhost/39015] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 16:54:07,512 INFO [Listener at localhost/39015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:35255 2023-02-09 16:54:07,522 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,524 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,544 INFO [Listener at localhost/39015] zookeeper.RecoverableZooKeeper(93): Process identifier=master:35255 connecting to ZooKeeper ensemble=127.0.0.1:59211 2023-02-09 16:54:07,585 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:352550x0, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 16:54:07,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:35255-0x1003a01243d0000 connected 2023-02-09 16:54:07,613 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 16:54:07,614 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:54:07,617 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 16:54:07,625 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35255 2023-02-09 16:54:07,626 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35255 2023-02-09 16:54:07,626 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35255 2023-02-09 16:54:07,626 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35255 2023-02-09 16:54:07,627 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35255 2023-02-09 16:54:07,633 INFO [Listener at localhost/39015] master.HMaster(439): hbase.rootdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9, hbase.cluster.distributed=false 2023-02-09 16:54:07,699 INFO [Listener at localhost/39015] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 16:54:07,699 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,699 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,699 INFO [Listener at localhost/39015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 16:54:07,699 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,699 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 16:54:07,704 INFO [Listener at localhost/39015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 16:54:07,707 INFO [Listener at localhost/39015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43347 2023-02-09 16:54:07,709 INFO [Listener at localhost/39015] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 16:54:07,715 DEBUG [Listener at localhost/39015] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 16:54:07,717 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,718 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,720 INFO [Listener at localhost/39015] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43347 connecting to ZooKeeper ensemble=127.0.0.1:59211 2023-02-09 16:54:07,727 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:433470x0, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 16:54:07,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43347-0x1003a01243d0001 connected 2023-02-09 16:54:07,728 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 16:54:07,729 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:54:07,730 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 16:54:07,731 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43347 2023-02-09 16:54:07,732 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43347 2023-02-09 16:54:07,732 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43347 2023-02-09 16:54:07,733 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43347 2023-02-09 16:54:07,733 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43347 2023-02-09 16:54:07,745 INFO [Listener at localhost/39015] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 16:54:07,746 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,746 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,746 INFO [Listener at localhost/39015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 16:54:07,746 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,747 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 16:54:07,747 INFO [Listener at localhost/39015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 16:54:07,748 INFO [Listener at localhost/39015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37073 2023-02-09 16:54:07,749 INFO [Listener at localhost/39015] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 16:54:07,750 DEBUG [Listener at localhost/39015] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 16:54:07,751 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,753 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,754 INFO [Listener at localhost/39015] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37073 connecting to ZooKeeper ensemble=127.0.0.1:59211 2023-02-09 16:54:07,759 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:370730x0, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 16:54:07,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37073-0x1003a01243d0002 connected 2023-02-09 16:54:07,760 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 16:54:07,761 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:54:07,762 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 16:54:07,762 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37073 2023-02-09 16:54:07,763 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37073 2023-02-09 16:54:07,763 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37073 2023-02-09 16:54:07,764 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37073 2023-02-09 16:54:07,764 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37073 2023-02-09 16:54:07,776 INFO [Listener at localhost/39015] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 16:54:07,776 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,776 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,776 INFO [Listener at localhost/39015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 16:54:07,776 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 16:54:07,777 INFO [Listener at localhost/39015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 16:54:07,777 INFO [Listener at localhost/39015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 16:54:07,778 INFO [Listener at localhost/39015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33383 2023-02-09 16:54:07,778 INFO [Listener at localhost/39015] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 16:54:07,779 DEBUG [Listener at localhost/39015] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 16:54:07,780 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,782 INFO [Listener at localhost/39015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,783 INFO [Listener at localhost/39015] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33383 connecting to ZooKeeper ensemble=127.0.0.1:59211 2023-02-09 16:54:07,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:333830x0, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 16:54:07,787 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33383-0x1003a01243d0003 connected 2023-02-09 16:54:07,787 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 16:54:07,788 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:54:07,789 DEBUG [Listener at localhost/39015] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 16:54:07,789 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33383 2023-02-09 16:54:07,789 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33383 2023-02-09 16:54:07,790 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33383 2023-02-09 16:54:07,790 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33383 2023-02-09 16:54:07,791 DEBUG [Listener at localhost/39015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33383 2023-02-09 16:54:07,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:07,801 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 16:54:07,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:07,821 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 16:54:07,821 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 16:54:07,821 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 16:54:07,821 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 16:54:07,822 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:07,823 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 16:54:07,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,35255,1675961646540 from backup master directory 2023-02-09 16:54:07,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 16:54:07,828 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:07,828 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 16:54:07,829 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 16:54:07,829 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:07,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 16:54:07,833 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 16:54:07,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/hbase.id with ID: 61591ce4-b653-4378-a74e-c33cd6552205 2023-02-09 16:54:07,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 16:54:07,976 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:08,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x2619c11d to 127.0.0.1:59211 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 16:54:08,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3363f0ed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 16:54:08,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 16:54:08,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 16:54:08,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 16:54:08,099 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 16:54:08,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:54:08,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:54:08,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 16:54:08,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store-tmp 2023-02-09 16:54:08,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:08,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 16:54:08,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:54:08,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:54:08,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 16:54:08,209 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:54:08,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:54:08,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675961648208Disabling compacts and flushes for region at 1675961648209 (+1 ms)Disabling writes for close at 1675961648209Writing region close event to WAL at 1675961648209Closed at 1675961648209 2023-02-09 16:54:08,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/WALs/jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:08,232 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C35255%2C1675961646540, suffix=, logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/WALs/jenkins-hbase4.apache.org,35255,1675961646540, archiveDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/oldWALs, maxLogs=10 2023-02-09 16:54:08,287 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK] 2023-02-09 16:54:08,287 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK] 2023-02-09 16:54:08,287 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK] 2023-02-09 16:54:08,295 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:54:08,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/WALs/jenkins-hbase4.apache.org,35255,1675961646540/jenkins-hbase4.apache.org%2C35255%2C1675961646540.1675961648243 2023-02-09 16:54:08,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK], DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK], DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK]] 2023-02-09 16:54:08,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:08,363 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:08,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 16:54:08,369 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 16:54:08,433 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 16:54:08,441 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 16:54:08,465 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 16:54:08,476 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:08,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 16:54:08,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 16:54:08,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 16:54:08,504 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:08,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64711008, jitterRate=-0.035730838775634766}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 16:54:08,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675961648404Initializing all the Stores at 1675961648406 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961648407 (+1 ms)Cleaning up temporary data from old regions at 1675961648492 (+85 ms)Cleaning up detritus from prior splits at 1675961648493 (+1 ms)Region opened successfully at 1675961648505 (+12 ms) 2023-02-09 16:54:08,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 16:54:08,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:08,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:08,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:08,529 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 16:54:08,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-09 16:54:08,561 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 16:54:08,593 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 16:54:08,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 16:54:08,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 16:54:08,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 16:54:08,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 16:54:08,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 16:54:08,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 16:54:08,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:08,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 16:54:08,652 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 16:54:08,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 16:54:08,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 16:54:08,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 16:54:08,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 16:54:08,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 16:54:08,670 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:08,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,35255,1675961646540, sessionid=0x1003a01243d0000, setting cluster-up flag (Was=false) 2023-02-09 16:54:08,685 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:08,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 16:54:08,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:08,697 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:08,703 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 16:54:08,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:08,706 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.hbase-snapshot/.tmp 2023-02-09 16:54:08,795 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(952): ClusterId : 61591ce4-b653-4378-a74e-c33cd6552205 2023-02-09 16:54:08,796 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(952): ClusterId : 61591ce4-b653-4378-a74e-c33cd6552205 2023-02-09 16:54:08,795 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(952): ClusterId : 61591ce4-b653-4378-a74e-c33cd6552205 2023-02-09 16:54:08,801 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 16:54:08,801 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 16:54:08,801 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 16:54:08,808 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 16:54:08,808 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 16:54:08,808 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 16:54:08,808 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 16:54:08,808 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 16:54:08,808 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 16:54:08,812 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 16:54:08,812 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 16:54:08,812 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 16:54:08,814 DEBUG [RS:0;jenkins-hbase4:43347] zookeeper.ReadOnlyZKClient(139): Connect 0x4fa7ac0a to 127.0.0.1:59211 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 16:54:08,814 DEBUG [RS:1;jenkins-hbase4:37073] zookeeper.ReadOnlyZKClient(139): Connect 0x11186348 to 127.0.0.1:59211 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 16:54:08,814 DEBUG [RS:2;jenkins-hbase4:33383] zookeeper.ReadOnlyZKClient(139): Connect 0x6afc572b to 127.0.0.1:59211 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 16:54:08,828 DEBUG [RS:0;jenkins-hbase4:43347] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41604b8e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 16:54:08,829 DEBUG [RS:2;jenkins-hbase4:33383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@643ba404, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 16:54:08,829 DEBUG [RS:0;jenkins-hbase4:43347] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@455fc77a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 16:54:08,829 DEBUG [RS:2;jenkins-hbase4:33383] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60043c5b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 16:54:08,829 DEBUG [RS:1;jenkins-hbase4:37073] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fdf9f1b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 16:54:08,830 DEBUG [RS:1;jenkins-hbase4:37073] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1be00838, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 16:54:08,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:08,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 16:54:08,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 16:54:08,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 16:54:08,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 16:54:08,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 16:54:08,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:08,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 16:54:08,842 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:08,843 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961678843 2023-02-09 16:54:08,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 16:54:08,851 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:08,851 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 16:54:08,855 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:33383 2023-02-09 16:54:08,857 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:37073 2023-02-09 16:54:08,857 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43347 2023-02-09 16:54:08,858 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 16:54:08,859 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 16:54:08,859 INFO [RS:2;jenkins-hbase4:33383] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 16:54:08,859 INFO [RS:1;jenkins-hbase4:37073] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 16:54:08,860 INFO [RS:1;jenkins-hbase4:37073] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 16:54:08,859 INFO [RS:0;jenkins-hbase4:43347] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 16:54:08,860 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 16:54:08,860 INFO [RS:2;jenkins-hbase4:33383] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 16:54:08,860 INFO [RS:0;jenkins-hbase4:43347] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 16:54:08,860 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 16:54:08,860 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 16:54:08,863 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35255,1675961646540 with isa=jenkins-hbase4.apache.org/172.31.14.131:43347, startcode=1675961647698 2023-02-09 16:54:08,863 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35255,1675961646540 with isa=jenkins-hbase4.apache.org/172.31.14.131:37073, startcode=1675961647745 2023-02-09 16:54:08,863 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,35255,1675961646540 with isa=jenkins-hbase4.apache.org/172.31.14.131:33383, startcode=1675961647775 2023-02-09 16:54:08,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 16:54:08,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 16:54:08,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 16:54:08,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 16:54:08,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:08,871 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 16:54:08,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 16:54:08,873 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 16:54:08,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 16:54:08,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 16:54:08,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675961648878,5,FailOnTimeoutGroup] 2023-02-09 16:54:08,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675961648880,5,FailOnTimeoutGroup] 2023-02-09 16:54:08,884 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:08,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 16:54:08,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:08,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:08,887 DEBUG [RS:1;jenkins-hbase4:37073] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 16:54:08,887 DEBUG [RS:0;jenkins-hbase4:43347] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 16:54:08,887 DEBUG [RS:2;jenkins-hbase4:33383] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 16:54:08,911 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:08,912 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:08,913 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9 2023-02-09 16:54:08,947 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:08,951 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 16:54:08,955 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/info 2023-02-09 16:54:08,956 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 16:54:08,957 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:08,957 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 16:54:08,959 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 16:54:08,960 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 16:54:08,961 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:08,961 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 16:54:08,963 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55193, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 16:54:08,963 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43877, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 16:54:08,963 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52257, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 16:54:08,964 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/table 2023-02-09 16:54:08,965 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 16:54:08,966 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:08,967 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740 2023-02-09 16:54:08,971 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740 2023-02-09 16:54:08,975 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 16:54:08,976 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35255] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:08,976 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:08,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=35255] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:08,978 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 16:54:08,983 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:08,984 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67278808, jitterRate=0.002532362937927246}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 16:54:08,984 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675961648948Initializing all the Stores at 1675961648949 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675961648950 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961648950Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675961648950Cleaning up temporary data from old regions at 1675961648973 (+23 ms)Cleaning up detritus from prior splits at 1675961648974 (+1 ms)Region opened successfully at 1675961648984 (+10 ms) 2023-02-09 16:54:08,984 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 16:54:08,984 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 16:54:08,984 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 16:54:08,984 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 16:54:08,984 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 16:54:08,985 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 16:54:08,985 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675961648984Disabling compacts and flushes for region at 1675961648984Disabling writes for close at 1675961648984Writing region close event to WAL at 1675961648985 (+1 ms)Closed at 1675961648985 2023-02-09 16:54:08,992 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:08,992 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 16:54:08,996 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9 2023-02-09 16:54:08,996 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9 2023-02-09 16:54:08,996 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9 2023-02-09 16:54:08,997 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44699 2023-02-09 16:54:08,997 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44699 2023-02-09 16:54:08,997 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 16:54:08,997 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44699 2023-02-09 16:54:08,997 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 16:54:08,997 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 16:54:09,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 16:54:09,007 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:54:09,008 DEBUG [RS:2;jenkins-hbase4:33383] zookeeper.ZKUtil(162): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,008 DEBUG [RS:0;jenkins-hbase4:43347] zookeeper.ZKUtil(162): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,009 WARN [RS:2;jenkins-hbase4:33383] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 16:54:09,009 WARN [RS:0;jenkins-hbase4:43347] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 16:54:09,010 DEBUG [RS:1;jenkins-hbase4:37073] zookeeper.ZKUtil(162): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,010 INFO [RS:0;jenkins-hbase4:43347] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 16:54:09,010 WARN [RS:1;jenkins-hbase4:37073] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 16:54:09,010 INFO [RS:2;jenkins-hbase4:33383] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 16:54:09,010 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,011 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,010 INFO [RS:1;jenkins-hbase4:37073] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 16:54:09,011 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33383,1675961647775] 2023-02-09 16:54:09,011 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43347,1675961647698] 2023-02-09 16:54:09,011 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,011 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37073,1675961647745] 2023-02-09 16:54:09,029 DEBUG [RS:1;jenkins-hbase4:37073] zookeeper.ZKUtil(162): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,029 DEBUG [RS:0;jenkins-hbase4:43347] zookeeper.ZKUtil(162): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,029 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 16:54:09,029 DEBUG [RS:2;jenkins-hbase4:33383] zookeeper.ZKUtil(162): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,029 DEBUG [RS:1;jenkins-hbase4:37073] zookeeper.ZKUtil(162): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,029 DEBUG [RS:0;jenkins-hbase4:43347] zookeeper.ZKUtil(162): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,029 DEBUG [RS:2;jenkins-hbase4:33383] zookeeper.ZKUtil(162): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,030 DEBUG [RS:1;jenkins-hbase4:37073] zookeeper.ZKUtil(162): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,030 DEBUG [RS:0;jenkins-hbase4:43347] zookeeper.ZKUtil(162): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,030 DEBUG [RS:2;jenkins-hbase4:33383] zookeeper.ZKUtil(162): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,031 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 16:54:09,041 DEBUG [RS:1;jenkins-hbase4:37073] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 16:54:09,041 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 16:54:09,041 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 16:54:09,051 INFO [RS:0;jenkins-hbase4:43347] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 16:54:09,051 INFO [RS:1;jenkins-hbase4:37073] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 16:54:09,051 INFO [RS:2;jenkins-hbase4:33383] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 16:54:09,072 INFO [RS:2;jenkins-hbase4:33383] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 16:54:09,072 INFO [RS:0;jenkins-hbase4:43347] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 16:54:09,072 INFO [RS:1;jenkins-hbase4:37073] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 16:54:09,076 INFO [RS:2;jenkins-hbase4:33383] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 16:54:09,076 INFO [RS:0;jenkins-hbase4:43347] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 16:54:09,076 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,076 INFO [RS:1;jenkins-hbase4:37073] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 16:54:09,076 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,077 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,077 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 16:54:09,077 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 16:54:09,077 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 16:54:09,086 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,086 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,086 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,086 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,086 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,086 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,086 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 16:54:09,087 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,087 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 16:54:09,087 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:0;jenkins-hbase4:43347] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:1;jenkins-hbase4:37073] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,088 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 16:54:09,089 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,089 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,089 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,089 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,089 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,089 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,089 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,089 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,090 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,090 DEBUG [RS:2;jenkins-hbase4:33383] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 16:54:09,092 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,092 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,092 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,111 INFO [RS:1;jenkins-hbase4:37073] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 16:54:09,111 INFO [RS:2;jenkins-hbase4:33383] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 16:54:09,111 INFO [RS:0;jenkins-hbase4:43347] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 16:54:09,113 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37073,1675961647745-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,113 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33383,1675961647775-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,113 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43347,1675961647698-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,132 INFO [RS:0;jenkins-hbase4:43347] regionserver.Replication(203): jenkins-hbase4.apache.org,43347,1675961647698 started 2023-02-09 16:54:09,132 INFO [RS:2;jenkins-hbase4:33383] regionserver.Replication(203): jenkins-hbase4.apache.org,33383,1675961647775 started 2023-02-09 16:54:09,132 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43347,1675961647698, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43347, sessionid=0x1003a01243d0001 2023-02-09 16:54:09,132 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33383,1675961647775, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33383, sessionid=0x1003a01243d0003 2023-02-09 16:54:09,132 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 16:54:09,132 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 16:54:09,132 DEBUG [RS:0;jenkins-hbase4:43347] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,133 DEBUG [RS:2;jenkins-hbase4:33383] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,133 DEBUG [RS:0;jenkins-hbase4:43347] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43347,1675961647698' 2023-02-09 16:54:09,133 DEBUG [RS:2;jenkins-hbase4:33383] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33383,1675961647775' 2023-02-09 16:54:09,134 DEBUG [RS:0;jenkins-hbase4:43347] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:09,134 DEBUG [RS:2;jenkins-hbase4:33383] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:09,135 INFO [RS:1;jenkins-hbase4:37073] regionserver.Replication(203): jenkins-hbase4.apache.org,37073,1675961647745 started 2023-02-09 16:54:09,135 DEBUG [RS:2;jenkins-hbase4:33383] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:09,135 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,37073,1675961647745, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37073, sessionid=0x1003a01243d0002 2023-02-09 16:54:09,135 DEBUG [RS:0;jenkins-hbase4:43347] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:09,135 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 16:54:09,135 DEBUG [RS:1;jenkins-hbase4:37073] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,135 DEBUG [RS:1;jenkins-hbase4:37073] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37073,1675961647745' 2023-02-09 16:54:09,135 DEBUG [RS:1;jenkins-hbase4:37073] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:09,135 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 16:54:09,135 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 16:54:09,135 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 16:54:09,136 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 16:54:09,136 DEBUG [RS:2;jenkins-hbase4:33383] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:09,136 DEBUG [RS:1;jenkins-hbase4:37073] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:09,136 DEBUG [RS:2;jenkins-hbase4:33383] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33383,1675961647775' 2023-02-09 16:54:09,136 DEBUG [RS:2;jenkins-hbase4:33383] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 16:54:09,136 DEBUG [RS:0;jenkins-hbase4:43347] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,137 DEBUG [RS:0;jenkins-hbase4:43347] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43347,1675961647698' 2023-02-09 16:54:09,137 DEBUG [RS:0;jenkins-hbase4:43347] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 16:54:09,137 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 16:54:09,137 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 16:54:09,137 DEBUG [RS:2;jenkins-hbase4:33383] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 16:54:09,137 DEBUG [RS:1;jenkins-hbase4:37073] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:09,137 DEBUG [RS:1;jenkins-hbase4:37073] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37073,1675961647745' 2023-02-09 16:54:09,137 DEBUG [RS:1;jenkins-hbase4:37073] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 16:54:09,137 DEBUG [RS:0;jenkins-hbase4:43347] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 16:54:09,137 DEBUG [RS:2;jenkins-hbase4:33383] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 16:54:09,138 INFO [RS:2;jenkins-hbase4:33383] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 16:54:09,138 INFO [RS:2;jenkins-hbase4:33383] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 16:54:09,138 DEBUG [RS:1;jenkins-hbase4:37073] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 16:54:09,138 DEBUG [RS:0;jenkins-hbase4:43347] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 16:54:09,138 INFO [RS:0;jenkins-hbase4:43347] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 16:54:09,139 INFO [RS:0;jenkins-hbase4:43347] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 16:54:09,140 DEBUG [RS:1;jenkins-hbase4:37073] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 16:54:09,140 INFO [RS:1;jenkins-hbase4:37073] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 16:54:09,140 INFO [RS:1;jenkins-hbase4:37073] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 16:54:09,183 DEBUG [jenkins-hbase4:35255] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-09 16:54:09,187 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:09,193 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:09,193 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:09,193 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:09,193 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:09,196 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43347,1675961647698, state=OPENING 2023-02-09 16:54:09,204 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 16:54:09,205 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:09,206 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 16:54:09,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:09,249 INFO [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43347%2C1675961647698, suffix=, logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,43347,1675961647698, archiveDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs, maxLogs=32 2023-02-09 16:54:09,249 INFO [RS:1;jenkins-hbase4:37073] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37073%2C1675961647745, suffix=, logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,37073,1675961647745, archiveDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs, maxLogs=32 2023-02-09 16:54:09,249 INFO [RS:2;jenkins-hbase4:33383] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33383%2C1675961647775, suffix=, logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,33383,1675961647775, archiveDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs, maxLogs=32 2023-02-09 16:54:09,271 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK] 2023-02-09 16:54:09,271 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK] 2023-02-09 16:54:09,275 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK] 2023-02-09 16:54:09,285 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK] 2023-02-09 16:54:09,285 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK] 2023-02-09 16:54:09,289 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK] 2023-02-09 16:54:09,290 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK] 2023-02-09 16:54:09,290 INFO [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,43347,1675961647698/jenkins-hbase4.apache.org%2C43347%2C1675961647698.1675961649253 2023-02-09 16:54:09,290 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK] 2023-02-09 16:54:09,291 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK] 2023-02-09 16:54:09,292 DEBUG [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK], DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK], DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK]] 2023-02-09 16:54:09,301 INFO [RS:2;jenkins-hbase4:33383] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,33383,1675961647775/jenkins-hbase4.apache.org%2C33383%2C1675961647775.1675961649253 2023-02-09 16:54:09,302 DEBUG [RS:2;jenkins-hbase4:33383] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK], DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK], DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK]] 2023-02-09 16:54:09,302 INFO [RS:1;jenkins-hbase4:37073] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,37073,1675961647745/jenkins-hbase4.apache.org%2C37073%2C1675961647745.1675961649253 2023-02-09 16:54:09,302 DEBUG [RS:1;jenkins-hbase4:37073] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK], DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK], DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK]] 2023-02-09 16:54:09,392 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:09,395 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 16:54:09,398 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36858, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 16:54:09,411 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 16:54:09,411 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 16:54:09,415 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43347%2C1675961647698.meta, suffix=.meta, logDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,43347,1675961647698, archiveDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs, maxLogs=32 2023-02-09 16:54:09,437 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK] 2023-02-09 16:54:09,439 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK] 2023-02-09 16:54:09,439 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK] 2023-02-09 16:54:09,447 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/WALs/jenkins-hbase4.apache.org,43347,1675961647698/jenkins-hbase4.apache.org%2C43347%2C1675961647698.meta.1675961649418.meta 2023-02-09 16:54:09,448 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40921,DS-ab703b6a-5980-4c81-86f5-c31e2a0ee2b2,DISK], DatanodeInfoWithStorage[127.0.0.1:42345,DS-b0a93b80-cb1b-4f93-a33d-cea81900ad8b,DISK], DatanodeInfoWithStorage[127.0.0.1:40493,DS-0bf6ed84-111e-4cb3-afb6-338abb4b8551,DISK]] 2023-02-09 16:54:09,448 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:09,452 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:09,453 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 16:54:09,471 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 16:54:09,473 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 16:54:09,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 16:54:09,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:09,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 16:54:09,478 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 16:54:09,481 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 16:54:09,483 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/info 2023-02-09 16:54:09,483 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/info 2023-02-09 16:54:09,484 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 16:54:09,485 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:09,485 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 16:54:09,486 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 16:54:09,487 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 16:54:09,487 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 16:54:09,488 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:09,488 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 16:54:09,490 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/table 2023-02-09 16:54:09,490 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/table 2023-02-09 16:54:09,490 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 16:54:09,491 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:09,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740 2023-02-09 16:54:09,496 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740 2023-02-09 16:54:09,499 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 16:54:09,501 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 16:54:09,502 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67462099, jitterRate=0.005263611674308777}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 16:54:09,503 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675961649479Writing region info on filesystem at 1675961649479Initializing all the Stores at 1675961649481 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675961649481Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961649481Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675961649481Cleaning up temporary data from old regions at 1675961649497 (+16 ms)Cleaning up detritus from prior splits at 1675961649498 (+1 ms)Running coprocessor post-open hooks at 1675961649502 (+4 ms)Region opened successfully at 1675961649502 2023-02-09 16:54:09,512 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675961649385 2023-02-09 16:54:09,532 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 16:54:09,533 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 16:54:09,533 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43347,1675961647698, state=OPEN 2023-02-09 16:54:09,537 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 16:54:09,537 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 16:54:09,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 16:54:09,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43347,1675961647698 in 328 msec 2023-02-09 16:54:09,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 16:54:09,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 540 msec 2023-02-09 16:54:09,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 798 msec 2023-02-09 16:54:09,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675961647816, completionTime=-1 2023-02-09 16:54:09,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-09 16:54:09,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:09,621 DEBUG [hconnection-0x14051d1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 16:54:09,625 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36864, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 16:54:09,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:09,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961709644 2023-02-09 16:54:09,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961769644 2023-02-09 16:54:09,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 91 msec 2023-02-09 16:54:09,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35255,1675961646540-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35255,1675961646540-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35255,1675961646540-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:35255, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,670 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 16:54:09,680 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 16:54:09,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 16:54:09,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 16:54:09,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:09,704 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:09,707 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:09,732 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:09,734 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66 empty. 2023-02-09 16:54:09,735 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:09,735 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 16:54:09,771 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:09,774 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2f94cc9ee2fe56d77bbf16d8818a2c66, NAME => 'hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:09,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:09,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 2f94cc9ee2fe56d77bbf16d8818a2c66, disabling compactions & flushes 2023-02-09 16:54:09,791 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:09,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:09,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. after waiting 0 ms 2023-02-09 16:54:09,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:09,791 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:09,791 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 2f94cc9ee2fe56d77bbf16d8818a2c66: Waiting for close lock at 1675961649791Disabling compacts and flushes for region at 1675961649791Disabling writes for close at 1675961649791Writing region close event to WAL at 1675961649791Closed at 1675961649791 2023-02-09 16:54:09,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:09,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675961649799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961649799"}]},"ts":"1675961649799"} 2023-02-09 16:54:09,842 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:09,844 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:09,849 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961649844"}]},"ts":"1675961649844"} 2023-02-09 16:54:09,854 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 16:54:09,859 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:09,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:09,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:09,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:09,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:09,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2f94cc9ee2fe56d77bbf16d8818a2c66, ASSIGN}] 2023-02-09 16:54:09,866 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2f94cc9ee2fe56d77bbf16d8818a2c66, ASSIGN 2023-02-09 16:54:09,867 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=2f94cc9ee2fe56d77bbf16d8818a2c66, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:10,020 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:10,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:10,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675961650021"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961650021"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961650021"}]},"ts":"1675961650021"} 2023-02-09 16:54:10,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 2f94cc9ee2fe56d77bbf16d8818a2c66, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:10,180 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:10,180 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 16:54:10,185 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41132, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 16:54:10,191 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:10,193 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2f94cc9ee2fe56d77bbf16d8818a2c66, NAME => 'hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:10,194 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:10,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:10,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,195 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,197 INFO [StoreOpener-2f94cc9ee2fe56d77bbf16d8818a2c66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,199 DEBUG [StoreOpener-2f94cc9ee2fe56d77bbf16d8818a2c66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/info 2023-02-09 16:54:10,199 DEBUG [StoreOpener-2f94cc9ee2fe56d77bbf16d8818a2c66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/info 2023-02-09 16:54:10,200 INFO [StoreOpener-2f94cc9ee2fe56d77bbf16d8818a2c66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f94cc9ee2fe56d77bbf16d8818a2c66 columnFamilyName info 2023-02-09 16:54:10,201 INFO [StoreOpener-2f94cc9ee2fe56d77bbf16d8818a2c66-1] regionserver.HStore(310): Store=2f94cc9ee2fe56d77bbf16d8818a2c66/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:10,202 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,203 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,207 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:54:10,210 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:10,211 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2f94cc9ee2fe56d77bbf16d8818a2c66; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71262189, jitterRate=0.06188936531543732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 16:54:10,211 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2f94cc9ee2fe56d77bbf16d8818a2c66: Running coprocessor pre-open hook at 1675961650195Writing region info on filesystem at 1675961650195Initializing all the Stores at 1675961650196 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675961650197 (+1 ms)Cleaning up temporary data from old regions at 1675961650204 (+7 ms)Cleaning up detritus from prior splits at 1675961650204Running coprocessor post-open hooks at 1675961650211 (+7 ms)Region opened successfully at 1675961650211 2023-02-09 16:54:10,214 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66., pid=6, masterSystemTime=1675961650180 2023-02-09 16:54:10,218 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:10,218 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:54:10,220 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:10,221 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675961650219"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961650219"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961650219"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961650219"}]},"ts":"1675961650219"} 2023-02-09 16:54:10,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 16:54:10,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 2f94cc9ee2fe56d77bbf16d8818a2c66, server=jenkins-hbase4.apache.org,33383,1675961647775 in 198 msec 2023-02-09 16:54:10,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 16:54:10,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=2f94cc9ee2fe56d77bbf16d8818a2c66, ASSIGN in 366 msec 2023-02-09 16:54:10,233 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:10,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961650233"}]},"ts":"1675961650233"} 2023-02-09 16:54:10,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 16:54:10,241 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:10,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 549 msec 2023-02-09 16:54:10,304 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 16:54:10,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 16:54:10,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:10,328 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 16:54:10,330 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41142, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 16:54:10,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:10,366 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 16:54:10,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 35 msec 2023-02-09 16:54:10,380 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:10,393 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 16:54:10,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-02-09 16:54:10,406 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 16:54:10,413 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 16:54:10,413 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.584sec 2023-02-09 16:54:10,416 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 16:54:10,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 16:54:10,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 16:54:10,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35255,1675961646540-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 16:54:10,421 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,35255,1675961646540-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 16:54:10,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 16:54:10,501 DEBUG [Listener at localhost/39015] zookeeper.ReadOnlyZKClient(139): Connect 0x37c223d6 to 127.0.0.1:59211 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 16:54:10,506 DEBUG [Listener at localhost/39015] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68a311bc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 16:54:10,520 DEBUG [hconnection-0x82ecaf6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 16:54:10,532 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36868, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 16:54:10,543 INFO [Listener at localhost/39015] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:54:10,578 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=363, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=243, ProcessCount=168, AvailableMemoryMB=3572 2023-02-09 16:54:10,583 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:10,584 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:10,584 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:10,592 DEBUG [Listener at localhost/39015] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 16:54:10,624 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 16:54:10,638 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:54:10,638 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:54:10,639 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:54:10,644 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:10,647 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:10,649 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:10,651 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:10,652 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda empty. 2023-02-09 16:54:10,654 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:10,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 16:54:10,682 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:10,685 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => a9a0852ba4a2981d6960c832753abdda, NAME => 'testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:10,715 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:10,715 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing a9a0852ba4a2981d6960c832753abdda, disabling compactions & flushes 2023-02-09 16:54:10,715 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:10,715 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:10,715 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. after waiting 0 ms 2023-02-09 16:54:10,715 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:10,715 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:10,715 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for a9a0852ba4a2981d6960c832753abdda: Waiting for close lock at 1675961650715Disabling compacts and flushes for region at 1675961650715Disabling writes for close at 1675961650715Writing region close event to WAL at 1675961650715Closed at 1675961650715 2023-02-09 16:54:10,720 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:10,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675961650729"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961650729"}]},"ts":"1675961650729"} 2023-02-09 16:54:10,733 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:10,734 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:10,734 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961650734"}]},"ts":"1675961650734"} 2023-02-09 16:54:10,737 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-09 16:54:10,743 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:10,745 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:10,745 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:10,745 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:10,746 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:10,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, ASSIGN}] 2023-02-09 16:54:10,750 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, ASSIGN 2023-02-09 16:54:10,752 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:54:10,903 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:10,904 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a9a0852ba4a2981d6960c832753abdda, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:10,904 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675961650904"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961650904"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961650904"}]},"ts":"1675961650904"} 2023-02-09 16:54:10,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a9a0852ba4a2981d6960c832753abdda, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:11,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9a0852ba4a2981d6960c832753abdda, NAME => 'testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:11,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:11,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:11,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,070 INFO [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,073 DEBUG [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1 2023-02-09 16:54:11,073 DEBUG [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1 2023-02-09 16:54:11,073 INFO [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9a0852ba4a2981d6960c832753abdda columnFamilyName cf1 2023-02-09 16:54:11,074 INFO [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] regionserver.HStore(310): Store=a9a0852ba4a2981d6960c832753abdda/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:11,074 INFO [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,076 DEBUG [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2 2023-02-09 16:54:11,077 DEBUG [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2 2023-02-09 16:54:11,077 INFO [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9a0852ba4a2981d6960c832753abdda columnFamilyName cf2 2023-02-09 16:54:11,078 INFO [StoreOpener-a9a0852ba4a2981d6960c832753abdda-1] regionserver.HStore(310): Store=a9a0852ba4a2981d6960c832753abdda/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:11,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:11,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:11,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:11,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a9a0852ba4a2981d6960c832753abdda; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66611643, jitterRate=-0.0074091702699661255}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:11,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a9a0852ba4a2981d6960c832753abdda: Running coprocessor pre-open hook at 1675961651069Writing region info on filesystem at 1675961651069Initializing all the Stores at 1675961651070 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961651070Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961651070Cleaning up temporary data from old regions at 1675961651083 (+13 ms)Cleaning up detritus from prior splits at 1675961651083Running coprocessor post-open hooks at 1675961651091 (+8 ms)Region opened successfully at 1675961651091 2023-02-09 16:54:11,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda., pid=11, masterSystemTime=1675961651061 2023-02-09 16:54:11,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,097 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a9a0852ba4a2981d6960c832753abdda, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,098 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675961651097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961651097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961651097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961651097"}]},"ts":"1675961651097"} 2023-02-09 16:54:11,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 16:54:11,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a9a0852ba4a2981d6960c832753abdda, server=jenkins-hbase4.apache.org,43347,1675961647698 in 193 msec 2023-02-09 16:54:11,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 16:54:11,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, ASSIGN in 358 msec 2023-02-09 16:54:11,109 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:11,109 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961651109"}]},"ts":"1675961651109"} 2023-02-09 16:54:11,111 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-09 16:54:11,114 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:11,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 472 msec 2023-02-09 16:54:11,169 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:11,170 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:11,173 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:11,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,181 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:11,181 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,182 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 16:54:11,182 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 16:54:11,183 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,183 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,185 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,185 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,185 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,185 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,186 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,186 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,188 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,190 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,190 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,190 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,190 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,191 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,191 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,191 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,191 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,192 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,192 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,192 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,192 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,192 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,193 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:11,193 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:11,193 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,193 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. started... 2023-02-09 16:54:11,195 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:54:11,195 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,197 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,197 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,197 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,198 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,198 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,302 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/6765236759a2443e80c18645e77f49d4 2023-02-09 16:54:11,390 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/4ed234e90961469cae3f16a499a439f9 2023-02-09 16:54:11,402 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/6765236759a2443e80c18645e77f49d4 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4 2023-02-09 16:54:11,413 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:11,416 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/4ed234e90961469cae3f16a499a439f9 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9 2023-02-09 16:54:11,427 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:11,431 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 236ms, sequenceid=6, compaction requested=false 2023-02-09 16:54:11,433 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:11,436 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9a0852ba4a2981d6960c832753abdda: 2023-02-09 16:54:11,436 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,436 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:11,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:11,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,441 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:11,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:11,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:11,443 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,443 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:11,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 16:54:11,447 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:11,447 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,447 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21e5bb05[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:11,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:11,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:11,454 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:11,454 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:11,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,454 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:11,454 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:11,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:11,455 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:11,460 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,460 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:11,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:11,461 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:11,461 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,461 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:11,461 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,462 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:11,462 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:11,463 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:11,462 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:11,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:11,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:11,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:11,472 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 16:54:11,472 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:11,472 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:11,472 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,472 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:11,475 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,475 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,475 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,475 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,475 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:11,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:11,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,479 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,480 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:11,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,480 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,481 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,482 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,485 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:11,485 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,486 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,501 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,500 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,501 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,501 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,501 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,501 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,501 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,501 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:11,501 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:11,501 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,502 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,502 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,502 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,505 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:11,505 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:11,605 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:11,606 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:11,606 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:11,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:11,637 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:11,637 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:11,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:11,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:11,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,642 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:11,642 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,643 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 16:54:11,643 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 16:54:11,643 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,643 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,645 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,646 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,647 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:11,648 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,648 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,648 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,648 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,648 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:11,649 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,649 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,648 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,649 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,649 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,649 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,649 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,650 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,650 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:11,650 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,650 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,650 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. started... 2023-02-09 16:54:11,650 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,650 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:11,651 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:11,652 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,652 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,653 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,654 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,654 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,654 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,678 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/ef126d56341c407aa30eea113638654c 2023-02-09 16:54:11,712 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/9c1c7262da944b46a9be5c829c0d04ba 2023-02-09 16:54:11,722 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/ef126d56341c407aa30eea113638654c as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c 2023-02-09 16:54:11,731 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:11,732 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/9c1c7262da944b46a9be5c829c0d04ba as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba 2023-02-09 16:54:11,740 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:11,742 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 92ms, sequenceid=19, compaction requested=false 2023-02-09 16:54:11,742 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9a0852ba4a2981d6960c832753abdda: 2023-02-09 16:54:11,742 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,742 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:11,743 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:11,743 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,743 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,743 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,745 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,745 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:11,745 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,745 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:11,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:11,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:11,748 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 16:54:11,748 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@347d7295[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:11,748 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:11,748 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:11,751 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:11,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,751 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:11,751 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:11,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,752 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:11,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,752 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:11,752 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:11,752 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:11,752 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:11,752 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,752 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:11,754 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,754 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:11,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:11,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:11,754 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,754 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:11,755 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:11,755 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,755 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:11,756 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:11,756 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:11,756 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:11,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:11,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:11,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:11,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:11,761 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 16:54:11,761 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:11,761 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:11,761 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,761 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:11,764 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:11,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:11,764 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,765 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,766 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:11,767 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,768 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:11,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,787 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:11,787 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:11,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:11,787 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:11,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:11,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:11,788 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:11,789 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,789 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,789 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,888 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:11,889 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:11,889 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:11,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:11,912 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:11,912 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:11,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:11,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:11,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,916 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:11,916 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,917 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 16:54:11,917 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 16:54:11,917 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,917 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,919 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,919 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,919 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:11,919 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:11,920 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,920 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:11,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:11,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:11,921 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:11,922 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,922 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,922 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,922 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:11,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,922 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:11,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,924 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:11,924 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,924 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,924 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,924 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,923 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:11,924 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. started... 2023-02-09 16:54:11,924 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:11,924 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:11,924 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:11,926 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,928 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,928 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,928 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,928 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:11,928 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:11,946 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/b5d5e120535a4c9e86af53b0ba062fa8 2023-02-09 16:54:11,970 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/9517a20a923b474da4812cce620e3972 2023-02-09 16:54:11,979 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/b5d5e120535a4c9e86af53b0ba062fa8 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8 2023-02-09 16:54:11,987 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:11,988 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/9517a20a923b474da4812cce620e3972 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972 2023-02-09 16:54:11,996 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:11,997 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 73ms, sequenceid=32, compaction requested=true 2023-02-09 16:54:11,997 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9a0852ba4a2981d6960c832753abdda: 2023-02-09 16:54:11,997 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:11,997 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:11,998 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:11,998 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:11,998 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:11,998 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,000 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,001 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,001 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 16:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@485432cd[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:12,003 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:12,004 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,006 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:12,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,005 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,006 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,005 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,006 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:12,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,006 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:12,006 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,006 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,006 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:12,009 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,009 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,009 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,011 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,011 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,011 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,011 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,011 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,011 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,012 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:12,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:12,017 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 16:54:12,017 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:12,017 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:12,017 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,017 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:12,018 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,018 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,018 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,018 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,019 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,020 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,020 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,023 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,023 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,023 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,027 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,027 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,027 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,027 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,027 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,028 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,028 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:12,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,028 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,028 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,028 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,028 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,028 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,028 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:12,029 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,130 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:12,130 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,130 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:12,154 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,154 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:12,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:12,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,159 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,159 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 16:54:12,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 16:54:12,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,162 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,162 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,162 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,162 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,163 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,164 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,165 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,165 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,165 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,165 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,166 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,167 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,168 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:12,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:12,168 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:12,168 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. started... 2023-02-09 16:54:12,168 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:12,170 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,171 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,171 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,172 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,172 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,172 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,192 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/99459ba0c0df471eb7de7fa4b022adc8 2023-02-09 16:54:12,214 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/f4f9d0f07dd4419598db8431a6582495 2023-02-09 16:54:12,223 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/99459ba0c0df471eb7de7fa4b022adc8 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8 2023-02-09 16:54:12,230 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:12,231 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/f4f9d0f07dd4419598db8431a6582495 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495 2023-02-09 16:54:12,238 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:12,239 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 71ms, sequenceid=45, compaction requested=true 2023-02-09 16:54:12,240 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9a0852ba4a2981d6960c832753abdda: 2023-02-09 16:54:12,240 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:12,240 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:12,240 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:12,240 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,241 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,241 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,245 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,245 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,246 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,246 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 16:54:12,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@417c65f0[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:12,249 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:12,249 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,251 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,252 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,251 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,252 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,252 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:12,252 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,256 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,256 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,257 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:12,261 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 16:54:12,261 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:12,261 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:12,261 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,261 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,263 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,264 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,265 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,265 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,267 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,267 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,271 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,271 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,271 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,272 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:12,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,273 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,273 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,273 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,273 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,273 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:12,273 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,274 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:12,274 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,274 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,374 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:12,374 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,374 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:12,409 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,410 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,411 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:12,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:12,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,414 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,414 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,415 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 16:54:12,415 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 16:54:12,416 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,416 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,417 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,417 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,417 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,421 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,421 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,421 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,421 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,421 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,423 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,423 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,424 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,424 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,424 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,424 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,424 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,428 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,428 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. started... 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:12,428 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,428 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:12,431 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,432 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,432 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,432 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,432 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,432 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,462 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 2023-02-09 16:54:12,512 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/5c85bbfd41f443e2b856c1b2734be613 2023-02-09 16:54:12,522 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 2023-02-09 16:54:12,531 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:12,532 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/5c85bbfd41f443e2b856c1b2734be613 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613 2023-02-09 16:54:12,543 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:12,545 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 117ms, sequenceid=58, compaction requested=true 2023-02-09 16:54:12,545 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9a0852ba4a2981d6960c832753abdda: 2023-02-09 16:54:12,545 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:12,545 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,548 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,549 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,549 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 16:54:12,551 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:12,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dcdd349[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:12,551 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,553 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,553 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,553 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:12,553 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,554 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:12,561 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,561 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,561 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,562 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,562 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,562 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,563 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,563 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,563 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,563 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:12,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:12,569 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 16:54:12,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:12,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:12,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,570 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:12,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,572 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,575 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,576 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,576 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,582 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,582 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,582 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,582 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,583 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,583 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,583 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,583 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:12,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,584 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:12,684 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:12,685 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,685 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:12,706 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,706 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:12,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:12,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,710 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,710 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 16:54:12,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 16:54:12,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,713 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,713 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,713 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:12,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,714 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,715 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,715 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 16:54:12,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,715 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,715 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,715 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:12,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,717 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 16:54:12,717 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,716 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:12,717 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,717 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. started... 2023-02-09 16:54:12,717 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,718 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:12,719 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,719 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,719 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,719 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,719 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,719 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,733 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/8658126d889e46649b968e9d72e12369 2023-02-09 16:54:12,756 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/73d4a3380f6847e9998edaa0f56554a5 2023-02-09 16:54:12,762 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/8658126d889e46649b968e9d72e12369 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369 2023-02-09 16:54:12,769 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:12,770 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/73d4a3380f6847e9998edaa0f56554a5 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5 2023-02-09 16:54:12,777 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:12,778 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 60ms, sequenceid=71, compaction requested=true 2023-02-09 16:54:12,779 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9a0852ba4a2981d6960c832753abdda: 2023-02-09 16:54:12,779 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:12,779 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:12,780 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:12,780 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,780 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 16:54:12,780 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 16:54:12,783 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:12,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 16:54:12,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@127be9bf[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:12,785 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:12,785 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,787 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,787 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:12,786 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:12,787 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:12,788 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,789 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,789 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,789 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 16:54:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,790 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 16:54:12,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:12,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:12,794 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 16:54:12,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:12,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:12,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,794 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,798 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:12,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,798 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,799 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,799 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:12,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,801 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:12,801 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,802 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,804 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:12,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,805 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,804 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 16:54:12,805 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,804 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:12,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:12,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:12,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,806 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 16:54:12,906 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:12,906 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:12,907 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:12,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:12,929 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,935 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:12,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:12,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:12,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:12,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, UNASSIGN}] 2023-02-09 16:54:12,945 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, UNASSIGN 2023-02-09 16:54:12,946 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a9a0852ba4a2981d6960c832753abdda, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:12,946 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675961652945"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961652945"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961652945"}]},"ts":"1675961652945"} 2023-02-09 16:54:12,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure a9a0852ba4a2981d6960c832753abdda, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:13,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a9a0852ba4a2981d6960c832753abdda, disabling compactions & flushes 2023-02-09 16:54:13,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:13,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:13,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. after waiting 0 ms 2023-02-09 16:54:13,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:13,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a9a0852ba4a2981d6960c832753abdda 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:54:13,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/b1e79ddc6911412a897ccddafdb1d4c2 2023-02-09 16:54:13,148 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/cf7aa90fe3224fca8a0083df4fa98e3f 2023-02-09 16:54:13,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf1/b1e79ddc6911412a897ccddafdb1d4c2 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2 2023-02-09 16:54:13,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:54:13,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.tmp/cf2/cf7aa90fe3224fca8a0083df4fa98e3f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f 2023-02-09 16:54:13,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:54:13,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 66ms, sequenceid=82, compaction requested=true 2023-02-09 16:54:13,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:54:13,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:13,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. 2023-02-09 16:54:13,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a9a0852ba4a2981d6960c832753abdda: Waiting for close lock at 1675961653107Running coprocessor pre-close hooks at 1675961653107Disabling compacts and flushes for region at 1675961653107Disabling writes for close at 1675961653107Obtaining lock to block concurrent updates at 1675961653108 (+1 ms)Preparing flush snapshotting stores in a9a0852ba4a2981d6960c832753abdda at 1675961653108Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675961653108Flushing stores of testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. at 1675961653108Flushing a9a0852ba4a2981d6960c832753abdda/cf1: creating writer at 1675961653109 (+1 ms)Flushing a9a0852ba4a2981d6960c832753abdda/cf1: appending metadata at 1675961653112 (+3 ms)Flushing a9a0852ba4a2981d6960c832753abdda/cf1: closing flushed file at 1675961653112Flushing a9a0852ba4a2981d6960c832753abdda/cf2: creating writer at 1675961653132 (+20 ms)Flushing a9a0852ba4a2981d6960c832753abdda/cf2: appending metadata at 1675961653135 (+3 ms)Flushing a9a0852ba4a2981d6960c832753abdda/cf2: closing flushed file at 1675961653135Flushing a9a0852ba4a2981d6960c832753abdda/cf1: reopening flushed file at 1675961653156 (+21 ms)Flushing a9a0852ba4a2981d6960c832753abdda/cf2: reopening flushed file at 1675961653166 (+10 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a9a0852ba4a2981d6960c832753abdda in 66ms, sequenceid=82, compaction requested=true at 1675961653173 (+7 ms)Writing region close event to WAL at 1675961653189 (+16 ms)Running coprocessor post-close hooks at 1675961653195 (+6 ms)Closed at 1675961653195 2023-02-09 16:54:13,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a9a0852ba4a2981d6960c832753abdda, regionState=CLOSED 2023-02-09 16:54:13,199 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675961653199"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961653199"}]},"ts":"1675961653199"} 2023-02-09 16:54:13,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 16:54:13,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure a9a0852ba4a2981d6960c832753abdda, server=jenkins-hbase4.apache.org,43347,1675961647698 in 253 msec 2023-02-09 16:54:13,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-09 16:54:13,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9a0852ba4a2981d6960c832753abdda, UNASSIGN in 260 msec 2023-02-09 16:54:13,224 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=a9a0852ba4a2981d6960c832753abdda, threads=14 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,225 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,226 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,227 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,227 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,228 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,228 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,228 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,253 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369, top=true 2023-02-09 16:54:13,255 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5, top=true 2023-02-09 16:54:13,256 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2, top=true 2023-02-09 16:54:13,259 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8, top=true 2023-02-09 16:54:13,261 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613, top=true 2023-02-09 16:54:13,267 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f, top=true 2023-02-09 16:54:13,340 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.splits/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2 for child: 5c7a74a2d1d2af537583d0338c22a787, parent: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,340 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,342 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.splits/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613 for child: 5c7a74a2d1d2af537583d0338c22a787, parent: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,342 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,344 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.splits/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5 for child: 5c7a74a2d1d2af537583d0338c22a787, parent: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,344 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.splits/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369 for child: 5c7a74a2d1d2af537583d0338c22a787, parent: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,344 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.splits/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f for child: 5c7a74a2d1d2af537583d0338c22a787, parent: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,344 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,344 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,344 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,346 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/.splits/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8 for child: 5c7a74a2d1d2af537583d0338c22a787, parent: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,346 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,428 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,428 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,437 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,447 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,448 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,728 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,804 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,805 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972 for region: a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:13,805 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region a9a0852ba4a2981d6960c832753abdda Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:54:13,859 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:54:13,861 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:54:13,864 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675961653864"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675961653864"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675961653864"}]},"ts":"1675961653864"} 2023-02-09 16:54:13,864 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961653864"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961653864"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961653864"}]},"ts":"1675961653864"} 2023-02-09 16:54:13,864 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961653864"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961653864"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961653864"}]},"ts":"1675961653864"} 2023-02-09 16:54:13,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, ASSIGN}] 2023-02-09 16:54:13,913 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, ASSIGN 2023-02-09 16:54:13,914 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:54:13,915 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, ASSIGN 2023-02-09 16:54:13,916 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:54:14,064 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:54:14,066 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=5c7a74a2d1d2af537583d0338c22a787, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,066 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961654066"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961654066"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961654066"}]},"ts":"1675961654066"} 2023-02-09 16:54:14,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 5c7a74a2d1d2af537583d0338c22a787, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:14,072 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=e30459e13786502111687c71f7099b6b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,072 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961654072"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961654072"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961654072"}]},"ts":"1675961654072"} 2023-02-09 16:54:14,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure e30459e13786502111687c71f7099b6b, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:14,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5c7a74a2d1d2af537583d0338c22a787, NAME => 'testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.', STARTKEY => '26', ENDKEY => ''} 2023-02-09 16:54:14,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:14,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:14,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,228 INFO [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,230 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1 2023-02-09 16:54:14,230 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1 2023-02-09 16:54:14,231 INFO [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5c7a74a2d1d2af537583d0338c22a787 columnFamilyName cf1 2023-02-09 16:54:14,262 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4-top 2023-02-09 16:54:14,268 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8-top 2023-02-09 16:54:14,275 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8-top 2023-02-09 16:54:14,280 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c-top 2023-02-09 16:54:14,286 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8 2023-02-09 16:54:14,291 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369 2023-02-09 16:54:14,295 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2 2023-02-09 16:54:14,296 INFO [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(310): Store=5c7a74a2d1d2af537583d0338c22a787/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:14,296 INFO [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,297 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2 2023-02-09 16:54:14,297 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2 2023-02-09 16:54:14,298 INFO [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5c7a74a2d1d2af537583d0338c22a787 columnFamilyName cf2 2023-02-09 16:54:14,312 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9-top 2023-02-09 16:54:14,317 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972-top 2023-02-09 16:54:14,322 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba-top 2023-02-09 16:54:14,327 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495-top 2023-02-09 16:54:14,332 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613 2023-02-09 16:54:14,337 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5 2023-02-09 16:54:14,341 DEBUG [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f 2023-02-09 16:54:14,342 INFO [StoreOpener-5c7a74a2d1d2af537583d0338c22a787-1] regionserver.HStore(310): Store=5c7a74a2d1d2af537583d0338c22a787/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:14,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:14,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:14,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5c7a74a2d1d2af537583d0338c22a787; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70601774, jitterRate=0.05204841494560242}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:14,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5c7a74a2d1d2af537583d0338c22a787: Running coprocessor pre-open hook at 1675961654227Writing region info on filesystem at 1675961654227Initializing all the Stores at 1675961654228 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961654228Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961654228Cleaning up temporary data from old regions at 1675961654345 (+117 ms)Cleaning up detritus from prior splits at 1675961654346 (+1 ms)Running coprocessor post-open hooks at 1675961654349 (+3 ms)Region opened successfully at 1675961654349 2023-02-09 16:54:14,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787., pid=17, masterSystemTime=1675961654221 2023-02-09 16:54:14,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:14,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:14,354 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:14,354 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:14,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,357 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e30459e13786502111687c71f7099b6b, NAME => 'testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.', STARTKEY => '', ENDKEY => '26'} 2023-02-09 16:54:14,357 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,357 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): 5c7a74a2d1d2af537583d0338c22a787/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:14,357 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): 5c7a74a2d1d2af537583d0338c22a787/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:14,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:14,357 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5c7a74a2d1d2af537583d0338c22a787/cf1 in testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,357 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5c7a74a2d1d2af537583d0338c22a787/cf2 in testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:14,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,357 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=5c7a74a2d1d2af537583d0338c22a787, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:14,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,358 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/.tmp, totalSize=34.9 K 2023-02-09 16:54:14,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961654357"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961654357"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961654357"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961654357"}]},"ts":"1675961654357"} 2023-02-09 16:54:14,358 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/.tmp, totalSize=34.9 K 2023-02-09 16:54:14,360 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961651150 2023-02-09 16:54:14,360 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961651150 2023-02-09 16:54:14,361 INFO [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,362 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961651614 2023-02-09 16:54:14,362 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961651614 2023-02-09 16:54:14,363 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961651891 2023-02-09 16:54:14,363 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961651891 2023-02-09 16:54:14,363 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1 2023-02-09 16:54:14,364 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1 2023-02-09 16:54:14,364 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961652132 2023-02-09 16:54:14,364 INFO [StoreOpener-e30459e13786502111687c71f7099b6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e30459e13786502111687c71f7099b6b columnFamilyName cf1 2023-02-09 16:54:14,364 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961652132 2023-02-09 16:54:14,364 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961652378 2023-02-09 16:54:14,366 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961652688 2023-02-09 16:54:14,366 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961652378 2023-02-09 16:54:14,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-09 16:54:14,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 5c7a74a2d1d2af537583d0338c22a787, server=jenkins-hbase4.apache.org,43347,1675961647698 in 294 msec 2023-02-09 16:54:14,367 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961652688 2023-02-09 16:54:14,367 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961652909 2023-02-09 16:54:14,369 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961652909 2023-02-09 16:54:14,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, ASSIGN in 454 msec 2023-02-09 16:54:14,387 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4-bottom 2023-02-09 16:54:14,397 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8-bottom 2023-02-09 16:54:14,404 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8-bottom 2023-02-09 16:54:14,405 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): 5c7a74a2d1d2af537583d0338c22a787#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:14,409 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5c7a74a2d1d2af537583d0338c22a787#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:14,412 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c-bottom 2023-02-09 16:54:14,416 INFO [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(310): Store=e30459e13786502111687c71f7099b6b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:14,416 INFO [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,417 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2 2023-02-09 16:54:14,417 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2 2023-02-09 16:54:14,418 INFO [StoreOpener-e30459e13786502111687c71f7099b6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e30459e13786502111687c71f7099b6b columnFamilyName cf2 2023-02-09 16:54:14,435 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9-bottom 2023-02-09 16:54:14,437 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/.tmp/cf1/dc4fb306c9094b8badeb9993ad679761 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/dc4fb306c9094b8badeb9993ad679761 2023-02-09 16:54:14,441 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/.tmp/cf2/80c096deb3014b87b07803babcf37d0b as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/80c096deb3014b87b07803babcf37d0b 2023-02-09 16:54:14,443 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972-bottom 2023-02-09 16:54:14,451 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba-bottom 2023-02-09 16:54:14,456 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5c7a74a2d1d2af537583d0338c22a787/cf1 of 5c7a74a2d1d2af537583d0338c22a787 into dc4fb306c9094b8badeb9993ad679761(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:14,457 DEBUG [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495-bottom 2023-02-09 16:54:14,456 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5c7a74a2d1d2af537583d0338c22a787/cf2 of 5c7a74a2d1d2af537583d0338c22a787 into 80c096deb3014b87b07803babcf37d0b(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:14,457 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5c7a74a2d1d2af537583d0338c22a787: 2023-02-09 16:54:14,457 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5c7a74a2d1d2af537583d0338c22a787: 2023-02-09 16:54:14,457 INFO [StoreOpener-e30459e13786502111687c71f7099b6b-1] regionserver.HStore(310): Store=e30459e13786502111687c71f7099b6b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:14,459 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787., storeName=5c7a74a2d1d2af537583d0338c22a787/cf1, priority=9, startTime=1675961654352; duration=0sec 2023-02-09 16:54:14,459 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787., storeName=5c7a74a2d1d2af537583d0338c22a787/cf2, priority=9, startTime=1675961654352; duration=0sec 2023-02-09 16:54:14,460 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:14,460 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:14,460 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:14,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e30459e13786502111687c71f7099b6b 2023-02-09 16:54:14,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e30459e13786502111687c71f7099b6b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61248745, jitterRate=-0.08732257783412933}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:14,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e30459e13786502111687c71f7099b6b: Running coprocessor pre-open hook at 1675961654358Writing region info on filesystem at 1675961654358Initializing all the Stores at 1675961654360 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961654360Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961654361 (+1 ms)Cleaning up temporary data from old regions at 1675961654463 (+102 ms)Cleaning up detritus from prior splits at 1675961654463Running coprocessor post-open hooks at 1675961654467 (+4 ms)Region opened successfully at 1675961654467 2023-02-09 16:54:14,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b., pid=18, masterSystemTime=1675961654221 2023-02-09 16:54:14,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:14,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:14,469 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 16:54:14,469 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 16:54:14,469 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,469 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): e30459e13786502111687c71f7099b6b/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:14,469 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,469 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e30459e13786502111687c71f7099b6b/cf1 in testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,469 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): e30459e13786502111687c71f7099b6b/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:14,469 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e30459e13786502111687c71f7099b6b/cf2 in testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,470 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/.tmp, totalSize=19.9 K 2023-02-09 16:54:14,470 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/.tmp, totalSize=19.9 K 2023-02-09 16:54:14,471 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961651150 2023-02-09 16:54:14,471 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961651150 2023-02-09 16:54:14,471 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961651614 2023-02-09 16:54:14,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:14,472 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961651614 2023-02-09 16:54:14,472 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961651891 2023-02-09 16:54:14,472 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=e30459e13786502111687c71f7099b6b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,473 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961654472"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961654472"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961654472"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961654472"}]},"ts":"1675961654472"} 2023-02-09 16:54:14,473 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961651891 2023-02-09 16:54:14,473 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961652132 2023-02-09 16:54:14,474 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961652132 2023-02-09 16:54:14,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-09 16:54:14,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure e30459e13786502111687c71f7099b6b, server=jenkins-hbase4.apache.org,43347,1675961647698 in 402 msec 2023-02-09 16:54:14,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-09 16:54:14,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, ASSIGN in 569 msec 2023-02-09 16:54:14,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 in 1.5550 sec 2023-02-09 16:54:14,519 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): e30459e13786502111687c71f7099b6b#cf1#compaction#16 average throughput is 0.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:14,520 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): e30459e13786502111687c71f7099b6b#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:14,543 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/.tmp/cf1/e13543771e304f3387a4fde9b46136e0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/e13543771e304f3387a4fde9b46136e0 2023-02-09 16:54:14,543 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/.tmp/cf2/5ba8dc1fe4c640688cbab90a68a0c403 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/5ba8dc1fe4c640688cbab90a68a0c403 2023-02-09 16:54:14,560 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in e30459e13786502111687c71f7099b6b/cf1 of e30459e13786502111687c71f7099b6b into e13543771e304f3387a4fde9b46136e0(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:14,560 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in e30459e13786502111687c71f7099b6b/cf2 of e30459e13786502111687c71f7099b6b into 5ba8dc1fe4c640688cbab90a68a0c403(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:14,560 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e30459e13786502111687c71f7099b6b: 2023-02-09 16:54:14,560 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e30459e13786502111687c71f7099b6b: 2023-02-09 16:54:14,560 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b., storeName=e30459e13786502111687c71f7099b6b/cf1, priority=12, startTime=1675961654468; duration=0sec 2023-02-09 16:54:14,560 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b., storeName=e30459e13786502111687c71f7099b6b/cf2, priority=12, startTime=1675961654469; duration=0sec 2023-02-09 16:54:14,560 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:14,560 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:14,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43347] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:36868 deadline: 1675961714687, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda. is not online on jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,821 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 16:54:14,834 INFO [Listener at localhost/39015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 16:54:14,837 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 16:54:14,874 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:14,875 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:14,875 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:14,886 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:14,887 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:14,887 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-09 16:54:14,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-09 16:54:14,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:14,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961654906"}]},"ts":"1675961654906"} 2023-02-09 16:54:14,908 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-09 16:54:14,910 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-09 16:54:14,910 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:14,910 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:14,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, UNASSIGN}] 2023-02-09 16:54:14,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 16:54:14,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, UNASSIGN 2023-02-09 16:54:14,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=5c7a74a2d1d2af537583d0338c22a787, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961654914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961654914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961654914"}]},"ts":"1675961654914"} 2023-02-09 16:54:14,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 5c7a74a2d1d2af537583d0338c22a787, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:14,920 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, UNASSIGN 2023-02-09 16:54:14,921 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=e30459e13786502111687c71f7099b6b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:14,921 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961654921"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961654921"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961654921"}]},"ts":"1675961654921"} 2023-02-09 16:54:14,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure e30459e13786502111687c71f7099b6b, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:15,017 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,017 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 16:54:15,021 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:54:15,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e30459e13786502111687c71f7099b6b 2023-02-09 16:54:15,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e30459e13786502111687c71f7099b6b, disabling compactions & flushes 2023-02-09 16:54:15,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:15,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:15,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. after waiting 0 ms 2023-02-09 16:54:15,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:15,079 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8-bottom] to archive 2023-02-09 16:54:15,080 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:15,088 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,091 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,093 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,095 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 16:54:15,095 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,096 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 16:54:15,122 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495-bottom] to archive 2023-02-09 16:54:15,123 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:15,125 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,127 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,128 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,129 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:54:15,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:15,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b. 2023-02-09 16:54:15,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e30459e13786502111687c71f7099b6b: Waiting for close lock at 1675961655073Running coprocessor pre-close hooks at 1675961655073Disabling compacts and flushes for region at 1675961655073Disabling writes for close at 1675961655073Writing region close event to WAL at 1675961655131 (+58 ms)Running coprocessor post-close hooks at 1675961655135 (+4 ms)Closed at 1675961655135 2023-02-09 16:54:15,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e30459e13786502111687c71f7099b6b 2023-02-09 16:54:15,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:15,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5c7a74a2d1d2af537583d0338c22a787, disabling compactions & flushes 2023-02-09 16:54:15,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:15,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:15,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. after waiting 0 ms 2023-02-09 16:54:15,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:15,138 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=e30459e13786502111687c71f7099b6b, regionState=CLOSED 2023-02-09 16:54:15,138 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961655138"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961655138"}]},"ts":"1675961655138"} 2023-02-09 16:54:15,139 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2] to archive 2023-02-09 16:54:15,140 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:15,143 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/6765236759a2443e80c18645e77f49d4.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-09 16:54:15,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure e30459e13786502111687c71f7099b6b, server=jenkins-hbase4.apache.org,43347,1675961647698 in 217 msec 2023-02-09 16:54:15,145 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/ef126d56341c407aa30eea113638654c.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e30459e13786502111687c71f7099b6b, UNASSIGN in 232 msec 2023-02-09 16:54:15,147 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/b5d5e120535a4c9e86af53b0ba062fa8.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,149 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/99459ba0c0df471eb7de7fa4b022adc8.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,150 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-3f6ce7e2ea42490eb3ce91825dacbda8 2023-02-09 16:54:15,152 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-8658126d889e46649b968e9d72e12369 2023-02-09 16:54:15,153 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-b1e79ddc6911412a897ccddafdb1d4c2 2023-02-09 16:54:15,156 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f] to archive 2023-02-09 16:54:15,156 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:15,159 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/4ed234e90961469cae3f16a499a439f9.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,160 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9c1c7262da944b46a9be5c829c0d04ba.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,162 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/9517a20a923b474da4812cce620e3972.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,164 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/f4f9d0f07dd4419598db8431a6582495.a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,165 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-5c85bbfd41f443e2b856c1b2734be613 2023-02-09 16:54:15,167 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-73d4a3380f6847e9998edaa0f56554a5 2023-02-09 16:54:15,168 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/testSplitTableRegionUnevenDaughter=a9a0852ba4a2981d6960c832753abdda-cf7aa90fe3224fca8a0083df4fa98e3f 2023-02-09 16:54:15,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:54:15,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:15,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787. 2023-02-09 16:54:15,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5c7a74a2d1d2af537583d0338c22a787: Waiting for close lock at 1675961655138Running coprocessor pre-close hooks at 1675961655138Disabling compacts and flushes for region at 1675961655138Disabling writes for close at 1675961655138Writing region close event to WAL at 1675961655169 (+31 ms)Running coprocessor post-close hooks at 1675961655174 (+5 ms)Closed at 1675961655175 (+1 ms) 2023-02-09 16:54:15,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:15,178 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=5c7a74a2d1d2af537583d0338c22a787, regionState=CLOSED 2023-02-09 16:54:15,178 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675961655178"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961655178"}]},"ts":"1675961655178"} 2023-02-09 16:54:15,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-09 16:54:15,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 5c7a74a2d1d2af537583d0338c22a787, server=jenkins-hbase4.apache.org,43347,1675961647698 in 263 msec 2023-02-09 16:54:15,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-09 16:54:15,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5c7a74a2d1d2af537583d0338c22a787, UNASSIGN in 271 msec 2023-02-09 16:54:15,187 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961655186"}]},"ts":"1675961655186"} 2023-02-09 16:54:15,188 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-09 16:54:15,191 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-09 16:54:15,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 298 msec 2023-02-09 16:54:15,219 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,219 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 16:54:15,221 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-09 16:54:15,222 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,222 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,227 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,236 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,238 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,239 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,239 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 16:54:15,246 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,246 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b 2023-02-09 16:54:15,246 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:15,250 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/recovered.edits] 2023-02-09 16:54:15,250 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/recovered.edits] 2023-02-09 16:54:15,250 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/recovered.edits] 2023-02-09 16:54:15,255 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/dc4fb306c9094b8badeb9993ad679761 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf1/dc4fb306c9094b8badeb9993ad679761 2023-02-09 16:54:15,255 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/e13543771e304f3387a4fde9b46136e0 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf1/e13543771e304f3387a4fde9b46136e0 2023-02-09 16:54:15,255 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/3f6ce7e2ea42490eb3ce91825dacbda8 2023-02-09 16:54:15,257 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/6765236759a2443e80c18645e77f49d4 2023-02-09 16:54:15,259 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/80c096deb3014b87b07803babcf37d0b to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/cf2/80c096deb3014b87b07803babcf37d0b 2023-02-09 16:54:15,259 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/8658126d889e46649b968e9d72e12369 2023-02-09 16:54:15,259 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/5ba8dc1fe4c640688cbab90a68a0c403 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/cf2/5ba8dc1fe4c640688cbab90a68a0c403 2023-02-09 16:54:15,261 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/99459ba0c0df471eb7de7fa4b022adc8 2023-02-09 16:54:15,264 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b1e79ddc6911412a897ccddafdb1d4c2 2023-02-09 16:54:15,264 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b/recovered.edits/90.seqid 2023-02-09 16:54:15,264 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787/recovered.edits/90.seqid 2023-02-09 16:54:15,265 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e30459e13786502111687c71f7099b6b 2023-02-09 16:54:15,265 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:15,266 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/b5d5e120535a4c9e86af53b0ba062fa8 2023-02-09 16:54:15,268 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf1/ef126d56341c407aa30eea113638654c 2023-02-09 16:54:15,271 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/4ed234e90961469cae3f16a499a439f9 2023-02-09 16:54:15,272 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/5c85bbfd41f443e2b856c1b2734be613 2023-02-09 16:54:15,274 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/73d4a3380f6847e9998edaa0f56554a5 2023-02-09 16:54:15,276 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9517a20a923b474da4812cce620e3972 2023-02-09 16:54:15,277 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/9c1c7262da944b46a9be5c829c0d04ba 2023-02-09 16:54:15,279 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/cf7aa90fe3224fca8a0083df4fa98e3f 2023-02-09 16:54:15,281 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/cf2/f4f9d0f07dd4419598db8431a6582495 2023-02-09 16:54:15,284 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/recovered.edits/85.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda/recovered.edits/85.seqid 2023-02-09 16:54:15,285 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9a0852ba4a2981d6960c832753abdda 2023-02-09 16:54:15,285 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 16:54:15,288 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,316 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-09 16:54:15,340 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,341 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 16:54:15,362 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-09 16:54:15,364 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,364 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-09 16:54:15,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961655364"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:15,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961655364"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:15,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961655364"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:15,367 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:54:15,367 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a9a0852ba4a2981d6960c832753abdda, NAME => 'testSplitTableRegionUnevenDaughter,,1675961650642.a9a0852ba4a2981d6960c832753abdda.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e30459e13786502111687c71f7099b6b, NAME => 'testSplitTableRegionUnevenDaughter,,1675961652929.e30459e13786502111687c71f7099b6b.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 5c7a74a2d1d2af537583d0338c22a787, NAME => 'testSplitTableRegionUnevenDaughter,26,1675961652929.5c7a74a2d1d2af537583d0338c22a787.', STARTKEY => '26', ENDKEY => ''}] 2023-02-09 16:54:15,367 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-09 16:54:15,367 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961655367"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:15,369 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-09 16:54:15,371 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:15,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 143 msec 2023-02-09 16:54:15,542 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,542 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 16:54:15,544 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-09 16:54:15,560 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=394 (was 363) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44344 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: (jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x14051d1-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60156 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x14051d1-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44258 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x14051d1-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43070 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44308 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60128 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44280 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60170 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60066 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:40802 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60148 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:40776 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44394 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44316 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:40932 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:40694 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x14051d1-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:43347-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=827 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=223 (was 243), ProcessCount=168 (was 168), AvailableMemoryMB=3456 (was 3572) 2023-02-09 16:54:15,574 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=394, OpenFileDescriptor=827, MaxFileDescriptor=60000, SystemLoadAverage=223, ProcessCount=168, AvailableMemoryMB=3455 2023-02-09 16:54:15,574 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:15,575 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:15,575 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:15,581 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:54:15,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:54:15,584 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:15,585 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:15,586 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:15,588 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,589 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a empty. 2023-02-09 16:54:15,589 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,589 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 16:54:15,601 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:15,602 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 17bc3062a59e914ed503fb224f492a7a, NAME => 'testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:15,614 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:15,614 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 17bc3062a59e914ed503fb224f492a7a, disabling compactions & flushes 2023-02-09 16:54:15,614 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,614 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,614 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. after waiting 0 ms 2023-02-09 16:54:15,614 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,614 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,614 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 17bc3062a59e914ed503fb224f492a7a: Waiting for close lock at 1675961655614Disabling compacts and flushes for region at 1675961655614Disabling writes for close at 1675961655614Writing region close event to WAL at 1675961655614Closed at 1675961655614 2023-02-09 16:54:15,617 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:15,618 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961655618"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961655618"}]},"ts":"1675961655618"} 2023-02-09 16:54:15,620 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:15,622 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:15,622 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961655622"}]},"ts":"1675961655622"} 2023-02-09 16:54:15,624 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-09 16:54:15,627 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:15,628 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:15,628 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:15,628 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:15,628 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:15,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, ASSIGN}] 2023-02-09 16:54:15,630 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, ASSIGN 2023-02-09 16:54:15,631 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:54:15,781 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:15,783 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=17bc3062a59e914ed503fb224f492a7a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:15,783 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961655783"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961655783"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961655783"}]},"ts":"1675961655783"} 2023-02-09 16:54:15,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:15,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 17bc3062a59e914ed503fb224f492a7a, NAME => 'testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:15,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:15,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:15,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,943 INFO [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,944 DEBUG [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1 2023-02-09 16:54:15,944 DEBUG [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1 2023-02-09 16:54:15,945 INFO [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 17bc3062a59e914ed503fb224f492a7a columnFamilyName cf1 2023-02-09 16:54:15,945 INFO [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] regionserver.HStore(310): Store=17bc3062a59e914ed503fb224f492a7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:15,945 INFO [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,947 DEBUG [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2 2023-02-09 16:54:15,947 DEBUG [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2 2023-02-09 16:54:15,949 INFO [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 17bc3062a59e914ed503fb224f492a7a columnFamilyName cf2 2023-02-09 16:54:15,950 INFO [StoreOpener-17bc3062a59e914ed503fb224f492a7a-1] regionserver.HStore(310): Store=17bc3062a59e914ed503fb224f492a7a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:15,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:15,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:15,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:15,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 17bc3062a59e914ed503fb224f492a7a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66194287, jitterRate=-0.013628259301185608}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:15,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 17bc3062a59e914ed503fb224f492a7a: Running coprocessor pre-open hook at 1675961655942Writing region info on filesystem at 1675961655942Initializing all the Stores at 1675961655942Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961655942Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961655943 (+1 ms)Cleaning up temporary data from old regions at 1675961655952 (+9 ms)Cleaning up detritus from prior splits at 1675961655952Running coprocessor post-open hooks at 1675961655958 (+6 ms)Region opened successfully at 1675961655958 2023-02-09 16:54:15,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a., pid=27, masterSystemTime=1675961655937 2023-02-09 16:54:15,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:15,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=17bc3062a59e914ed503fb224f492a7a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:15,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961655962"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961655962"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961655962"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961655962"}]},"ts":"1675961655962"} 2023-02-09 16:54:15,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-09 16:54:15,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 in 179 msec 2023-02-09 16:54:15,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-09 16:54:15,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, ASSIGN in 339 msec 2023-02-09 16:54:15,970 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:15,970 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961655970"}]},"ts":"1675961655970"} 2023-02-09 16:54:15,972 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-09 16:54:15,975 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:15,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 393 msec 2023-02-09 16:54:16,094 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,095 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,095 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:16,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 16:54:16,097 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 16:54:16,097 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,098 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 16:54:16,098 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 16:54:16,099 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,099 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,100 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,100 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,100 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,100 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,101 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,101 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,102 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,103 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,102 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,103 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,103 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:16,104 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,105 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,105 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. started... 2023-02-09 16:54:16,105 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,105 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:54:16,107 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,107 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,107 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,107 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,108 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,108 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,118 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 2023-02-09 16:54:16,134 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/abc6f97b082b4b829e4112ec98cd710d 2023-02-09 16:54:16,140 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 2023-02-09 16:54:16,145 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:16,146 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/abc6f97b082b4b829e4112ec98cd710d as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d 2023-02-09 16:54:16,150 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:16,151 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 46ms, sequenceid=6, compaction requested=false 2023-02-09 16:54:16,151 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-09 16:54:16,152 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 17bc3062a59e914ed503fb224f492a7a: 2023-02-09 16:54:16,152 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,152 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:16,152 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:16,152 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,152 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,152 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,155 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,155 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,155 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 16:54:16,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bd07367[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:16,157 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 16:54:16,157 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,158 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,158 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,159 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:16,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:16,159 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:16,161 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,161 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,161 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,162 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,162 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,162 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,162 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,162 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,162 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,163 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:16,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 16:54:16,166 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 16:54:16,166 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:16,166 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:16,166 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 16:54:16,167 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,170 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,170 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,171 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,172 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,173 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,173 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,175 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,175 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,175 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,175 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,176 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,175 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,176 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,176 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,175 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 16:54:16,177 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:16,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,176 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,177 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:16,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,176 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,177 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:16,177 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,177 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,178 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,178 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,178 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,178 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,178 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,277 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:16,278 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,278 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:16,295 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,295 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:16,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:16,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 16:54:16,297 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 16:54:16,297 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,298 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 16:54:16,298 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 16:54:16,299 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,299 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,300 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,300 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,300 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,300 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,300 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,300 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,302 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,302 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,302 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,302 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,302 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,303 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,304 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:16,305 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:16,305 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,305 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. started... 2023-02-09 16:54:16,305 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,306 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,317 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/f82f62afc5be4bd395f0ab516b66f05c 2023-02-09 16:54:16,331 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/7f285d8c881948f28299fc457a9fb4f2 2023-02-09 16:54:16,337 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/f82f62afc5be4bd395f0ab516b66f05c as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c 2023-02-09 16:54:16,341 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:16,342 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/7f285d8c881948f28299fc457a9fb4f2 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 2023-02-09 16:54:16,347 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:16,348 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 43ms, sequenceid=19, compaction requested=false 2023-02-09 16:54:16,348 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 17bc3062a59e914ed503fb224f492a7a: 2023-02-09 16:54:16,348 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,351 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,351 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,351 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,351 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 16:54:16,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59a1ca08[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:16,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 16:54:16,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,355 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:16,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,355 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:16,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,365 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,365 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,365 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,365 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,365 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,365 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,366 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:16,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 16:54:16,370 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 16:54:16,370 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:16,370 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:16,370 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 16:54:16,371 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:16,372 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,372 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,372 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,373 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,373 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,373 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,374 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,376 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,376 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,376 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,379 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,379 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,379 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,379 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,380 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 16:54:16,380 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,380 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,381 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,381 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:16,381 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,381 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:16,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,381 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,381 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,481 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:16,481 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,482 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:16,499 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,499 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,500 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:16,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:16,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 16:54:16,502 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 16:54:16,502 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,503 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 16:54:16,503 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 16:54:16,504 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,504 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,506 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,506 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,506 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,506 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,506 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,507 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,508 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,508 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,508 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,508 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,509 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,509 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,509 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,517 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,517 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,519 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,519 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,520 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,520 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,520 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:16,520 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:16,520 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,520 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. started... 2023-02-09 16:54:16,520 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:16,522 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,522 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,522 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,523 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,523 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,523 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,531 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/229965d9e893483ea8a4d07f032d95cb 2023-02-09 16:54:16,547 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/708d8023a2664abf80e51ebea3a74c41 2023-02-09 16:54:16,553 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/229965d9e893483ea8a4d07f032d95cb as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb 2023-02-09 16:54:16,558 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:16,558 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/708d8023a2664abf80e51ebea3a74c41 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 2023-02-09 16:54:16,563 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:16,564 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 44ms, sequenceid=32, compaction requested=true 2023-02-09 16:54:16,564 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 17bc3062a59e914ed503fb224f492a7a: 2023-02-09 16:54:16,564 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,564 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:16,564 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:16,564 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,567 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,567 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,567 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,567 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 16:54:16,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ad50dee[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:16,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 16:54:16,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,573 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,573 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:16,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,573 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:16,574 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:16,574 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,574 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,574 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:16,575 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,577 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,577 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,577 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,577 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,578 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,577 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:16,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 16:54:16,583 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 16:54:16,583 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:16,583 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:16,583 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 16:54:16,583 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,586 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,587 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,587 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,589 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,590 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,590 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,594 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 16:54:16,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:16,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,595 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,595 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:16,596 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:16,595 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,596 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,596 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,596 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,596 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,596 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,696 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:16,696 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,696 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:16,699 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 16:54:16,700 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 16:54:16,700 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:16,700 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-09 16:54:16,714 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,715 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:16,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:16,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 16:54:16,717 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 16:54:16,717 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,718 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 16:54:16,718 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 16:54:16,719 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,719 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,720 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,720 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,720 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:16,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,722 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,722 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,723 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,723 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,723 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,723 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:16,724 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,723 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,724 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,724 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. started... 2023-02-09 16:54:16,724 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:16,724 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,724 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,724 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,725 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,726 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,726 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,724 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:16,731 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,731 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,732 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,732 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,732 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,732 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,738 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/63db2fd56776442d8bd7f50f55213904 2023-02-09 16:54:16,751 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/0edf2c81cb3d44da92f452c523b7d026 2023-02-09 16:54:16,756 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/63db2fd56776442d8bd7f50f55213904 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 2023-02-09 16:54:16,761 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:16,761 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/0edf2c81cb3d44da92f452c523b7d026 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 2023-02-09 16:54:16,766 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:16,767 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 43ms, sequenceid=45, compaction requested=true 2023-02-09 16:54:16,767 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 17bc3062a59e914ed503fb224f492a7a: 2023-02-09 16:54:16,767 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,767 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,768 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,769 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,770 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 16:54:16,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@372e251a[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:16,772 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 16:54:16,772 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,773 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,773 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,773 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,774 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,774 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:16,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:16,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,774 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:16,776 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,776 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,776 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,776 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,777 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,776 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,778 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,778 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,778 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,778 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:16,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 16:54:16,783 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 16:54:16,783 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:16,783 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:16,783 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 16:54:16,783 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,786 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,785 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,786 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,788 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,790 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,790 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,790 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,794 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 16:54:16,794 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:16,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,795 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:16,795 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:16,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,796 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,895 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:16,896 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,896 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:16,915 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:16,915 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:16,916 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:16,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:16,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 16:54:16,918 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 16:54:16,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 16:54:16,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 16:54:16,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,920 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,920 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,920 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,920 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,921 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,921 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:16,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,922 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,922 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,922 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,922 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,922 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:16,923 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:16,923 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,923 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. started... 2023-02-09 16:54:16,923 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,923 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:16,924 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,924 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,925 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,925 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,925 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,925 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,939 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/9fc6980fd4d344f2abe403520675e1be 2023-02-09 16:54:16,956 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/8ddb0f46753e437eafa8b3b79209cf78 2023-02-09 16:54:16,962 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/9fc6980fd4d344f2abe403520675e1be as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be 2023-02-09 16:54:16,967 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:16,968 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/8ddb0f46753e437eafa8b3b79209cf78 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 2023-02-09 16:54:16,973 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:16,974 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 51ms, sequenceid=58, compaction requested=true 2023-02-09 16:54:16,974 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 17bc3062a59e914ed503fb224f492a7a: 2023-02-09 16:54:16,975 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:16,975 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:16,975 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:16,975 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,975 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:16,975 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:16,976 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,977 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 16:54:16,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f9d33d3[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:16,979 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 16:54:16,979 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,981 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,981 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:16,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:16,981 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,984 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:16,984 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:16,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:16,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 16:54:16,987 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 16:54:16,987 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:16,987 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:16,987 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 16:54:16,987 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:16,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,989 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:16,989 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:16,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,992 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:16,992 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:16,992 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:16,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:16,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:16,998 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,998 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,998 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:16,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:16,999 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,999 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,999 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:16,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,000 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 16:54:17,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,000 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,001 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:17,001 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:17,101 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:17,101 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:17,101 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:17,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:17,119 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:17,119 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:17,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:17,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:17,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 16:54:17,122 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 16:54:17,122 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:17,122 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 16:54:17,122 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 16:54:17,122 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,123 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,124 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,124 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:17,124 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:17,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:17,124 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:17,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:17,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:17,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:17,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:17,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:17,125 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,125 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:17,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:17,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 16:54:17,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:17,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:17,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:17,127 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:17,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:17,127 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:17,127 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:17,128 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:17,128 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:17,128 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:17,128 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. started... 2023-02-09 16:54:17,128 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:17,128 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,129 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,129 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:17,130 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,130 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,130 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:17,144 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/1456800b4cb748fd83e27a0ee45cfd7a 2023-02-09 16:54:17,161 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/cee9b13fe78941cca81787dcdb88c45e 2023-02-09 16:54:17,167 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/1456800b4cb748fd83e27a0ee45cfd7a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a 2023-02-09 16:54:17,172 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:17,173 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/cee9b13fe78941cca81787dcdb88c45e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e 2023-02-09 16:54:17,179 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:17,180 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 52ms, sequenceid=71, compaction requested=true 2023-02-09 16:54:17,180 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 17bc3062a59e914ed503fb224f492a7a: 2023-02-09 16:54:17,180 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:17,180 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:17,180 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:17,180 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,181 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 16:54:17,181 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 16:54:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,182 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:17,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 16:54:17,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@183b41a2[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:17,185 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 16:54:17,185 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,192 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:17,192 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:17,192 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,193 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:17,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:17,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:17,193 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:17,194 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:17,194 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:17,194 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 16:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:17,195 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:17,195 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 16:54:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:17,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:54:17,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 16:54:17,199 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 16:54:17,199 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:17,199 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:17,199 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 16:54:17,199 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:17,200 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,200 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,200 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:17,200 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:17,200 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:17,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,200 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:17,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,201 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,201 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:17,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,202 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:17,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,202 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:17,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,204 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:17,204 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 16:54:17,205 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:17,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:17,208 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 16:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:17,208 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:17,208 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:17,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:17,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:17,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:17,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:17,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,209 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 16:54:17,210 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 16:54:17,309 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:17,309 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:17,309 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:17,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:17,574 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 16:54:17,575 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:17,576 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,577 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,578 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:17,578 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:17,579 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:17,826 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,827 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:17,828 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:17,828 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:17,830 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-09 16:54:17,830 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:17,830 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:17,830 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:17,830 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:17,831 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:17,834 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 16:54:17,835 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:17,836 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:17,836 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:17,837 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:17,837 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:17,837 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:17,838 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:17,838 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:17,839 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:17,839 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,839 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:17,839 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,840 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,840 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,840 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 16:54:17,841 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:17,841 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:17,842 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961687842 2023-02-09 16:54:17,843 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,843 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:17,847 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:17,847 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:17,847 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:17,848 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961717848 2023-02-09 16:54:17,848 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961777848 2023-02-09 16:54:17,848 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:54:17,848 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:17,850 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:17,850 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:17,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:17,850 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:18,098 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,098 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:18,098 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:18,098 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:18,100 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:18,100 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:18,100 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:18,100 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:18,100 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:18,101 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:18,102 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 16:54:18,103 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:18,104 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:18,104 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:18,104 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:18,104 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,105 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:18,105 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,105 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,106 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:18,106 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,106 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,106 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,106 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,106 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,107 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:18,107 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:18,108 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:18,108 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961688108 2023-02-09 16:54:18,110 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,110 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:18,113 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:18,113 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:18,114 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:18,114 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961718114 2023-02-09 16:54:18,114 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961778114 2023-02-09 16:54:18,114 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:54:18,114 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:18,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN}] 2023-02-09 16:54:18,117 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:18,117 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:18,117 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:18,364 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,364 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:18,364 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:18,365 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:18,366 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:18,366 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:18,366 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:18,366 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:18,366 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:18,367 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:18,368 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 16:54:18,368 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:18,369 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:18,369 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:18,369 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:18,370 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,370 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:18,370 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,370 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,371 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:18,371 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,371 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,371 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,371 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,371 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,372 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:18,372 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:18,373 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:18,373 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961688373 2023-02-09 16:54:18,374 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:18,375 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,377 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:18,377 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:18,378 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:18,378 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961718378 2023-02-09 16:54:18,378 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961778378 2023-02-09 16:54:18,378 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:18,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN}] 2023-02-09 16:54:18,378 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:18,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,381 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:18,381 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961658381"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961658381"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961658381"}]},"ts":"1675961658381"} 2023-02-09 16:54:18,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:18,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:18,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:18,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:18,628 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,628 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:18,628 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:18,628 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:18,629 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:18,629 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:18,629 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:18,629 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:18,630 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:18,630 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:18,632 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:18,632 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 16:54:18,632 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:18,632 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:18,633 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:18,633 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:18,633 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,634 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:18,634 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,634 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,634 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:18,635 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,635 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,636 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,636 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,636 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,636 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,636 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,636 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,636 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:18,637 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:18,637 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:18,637 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a to restore RIT 2023-02-09 16:54:18,638 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961688638 2023-02-09 16:54:18,638 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:18,639 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,641 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:18,641 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:18,641 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:18,641 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961718641 2023-02-09 16:54:18,641 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961778641 2023-02-09 16:54:18,641 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:18,642 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:18,642 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961658642"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961658642"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961658642"}]},"ts":"1675961658642"} 2023-02-09 16:54:18,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:18,644 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:18,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:18,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 17bc3062a59e914ed503fb224f492a7a, disabling compactions & flushes 2023-02-09 16:54:18,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:18,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:18,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. after waiting 0 ms 2023-02-09 16:54:18,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:18,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 17bc3062a59e914ed503fb224f492a7a 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:54:18,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/b54e4237840e41799c443137acffeec7 2023-02-09 16:54:18,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 2023-02-09 16:54:18,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf1/b54e4237840e41799c443137acffeec7 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 2023-02-09 16:54:18,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:54:18,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/.tmp/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 2023-02-09 16:54:18,842 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:54:18,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 45ms, sequenceid=82, compaction requested=true 2023-02-09 16:54:18,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:54:18,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:18,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. 2023-02-09 16:54:18,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 17bc3062a59e914ed503fb224f492a7a: Waiting for close lock at 1675961658798Running coprocessor pre-close hooks at 1675961658798Disabling compacts and flushes for region at 1675961658798Disabling writes for close at 1675961658798Obtaining lock to block concurrent updates at 1675961658798Preparing flush snapshotting stores in 17bc3062a59e914ed503fb224f492a7a at 1675961658798Finished memstore snapshotting testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675961658798Flushing stores of testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. at 1675961658799 (+1 ms)Flushing 17bc3062a59e914ed503fb224f492a7a/cf1: creating writer at 1675961658800 (+1 ms)Flushing 17bc3062a59e914ed503fb224f492a7a/cf1: appending metadata at 1675961658802 (+2 ms)Flushing 17bc3062a59e914ed503fb224f492a7a/cf1: closing flushed file at 1675961658802Flushing 17bc3062a59e914ed503fb224f492a7a/cf2: creating writer at 1675961658813 (+11 ms)Flushing 17bc3062a59e914ed503fb224f492a7a/cf2: appending metadata at 1675961658816 (+3 ms)Flushing 17bc3062a59e914ed503fb224f492a7a/cf2: closing flushed file at 1675961658816Flushing 17bc3062a59e914ed503fb224f492a7a/cf1: reopening flushed file at 1675961658832 (+16 ms)Flushing 17bc3062a59e914ed503fb224f492a7a/cf2: reopening flushed file at 1675961658838 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 17bc3062a59e914ed503fb224f492a7a in 45ms, sequenceid=82, compaction requested=true at 1675961658843 (+5 ms)Writing region close event to WAL at 1675961658860 (+17 ms)Running coprocessor post-close hooks at 1675961658863 (+3 ms)Closed at 1675961658864 (+1 ms) 2023-02-09 16:54:18,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:18,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED 2023-02-09 16:54:18,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961658867"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961658867"}]},"ts":"1675961658867"} 2023-02-09 16:54:18,869 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:18,869 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:18,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:18,869 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:18,892 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:18,892 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:18,893 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:18,894 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:18,894 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 16:54:18,894 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 16:54:18,895 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:18,895 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:18,895 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:18,896 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:18,896 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,896 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:18,896 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,897 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:18,897 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:18,898 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,898 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,898 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:18,898 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,898 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,898 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,898 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:18,898 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,898 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,898 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:18,898 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:18,899 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:18,899 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:18,900 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a to restore RIT 2023-02-09 16:54:18,900 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961688900 2023-02-09 16:54:18,901 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:18,901 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:18,903 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:18,903 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:18,903 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:18,903 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961718903 2023-02-09 16:54:18,903 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961778903 2023-02-09 16:54:18,903 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:54:18,904 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:18,904 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961658904"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961658904"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961658904"}]},"ts":"1675961658904"} 2023-02-09 16:54:18,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:18,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-09 16:54:18,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 in 261 msec 2023-02-09 16:54:18,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:18,908 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:18,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:18,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:19,154 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:19,154 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:19,155 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:19,156 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:19,157 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 16:54:19,157 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:19,157 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:19,158 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:19,158 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:19,158 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:19,158 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,159 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:19,159 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,159 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,159 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:19,160 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,160 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:19,160 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:19,160 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:19,160 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,160 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,160 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,160 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,160 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,161 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:19,161 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:19,162 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:19,162 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a to restore RIT 2023-02-09 16:54:19,162 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961689162 2023-02-09 16:54:19,162 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:19,163 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:19,166 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:19,166 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:19,166 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:19,166 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961719166 2023-02-09 16:54:19,166 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961779166 2023-02-09 16:54:19,166 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:54:19,166 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:19,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:19,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-09 16:54:19,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN in 790 msec 2023-02-09 16:54:19,174 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:19,174 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:19,174 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:19,416 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,416 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:19,417 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:19,417 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:19,418 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:19,418 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:19,418 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:19,418 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:19,418 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:19,419 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:19,420 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:19,420 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:19,420 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:19,420 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:19,421 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:19,421 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:19,421 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:19,421 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,422 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:19,422 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,422 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,422 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:19,423 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,423 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:19,423 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:19,423 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,423 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,423 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,423 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:19,424 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:19,424 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:19,425 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961689425 2023-02-09 16:54:19,425 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:19,426 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,428 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:19,429 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:19,429 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:19,429 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961719429 2023-02-09 16:54:19,429 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961779429 2023-02-09 16:54:19,429 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:54:19,429 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,429 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,429 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:19,443 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=17bc3062a59e914ed503fb224f492a7a, threads=14 2023-02-09 16:54:19,444 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,444 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,444 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,444 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,444 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,444 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,445 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,445 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,445 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,445 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,445 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,448 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,448 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,448 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,565 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,578 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,578 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,580 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,582 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,585 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,593 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,595 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,596 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,605 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,610 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,610 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,610 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,612 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,612 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 17bc3062a59e914ed503fb224f492a7a Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:54:19,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:19,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:19,658 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:19,679 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,679 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:19,679 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:19,679 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:19,680 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:19,681 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:19,681 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:19,681 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:19,681 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:19,681 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:19,682 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 16:54:19,682 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:19,682 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:19,683 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:19,683 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:19,684 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:19,684 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:19,684 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,685 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:19,685 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,685 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:19,686 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:19,687 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,687 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:19,687 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:19,687 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:19,687 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,687 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,687 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,687 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,687 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:19,687 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:19,688 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:19,688 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:19,689 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961689689 2023-02-09 16:54:19,690 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:19,690 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:19,693 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:19,693 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:19,693 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:19,693 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961719693 2023-02-09 16:54:19,693 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961779693 2023-02-09 16:54:19,693 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:54:19,703 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=17bc3062a59e914ed503fb224f492a7a, threads=14 2023-02-09 16:54:19,704 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,706 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,706 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,706 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,707 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,707 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,707 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,707 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,708 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,712 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,715 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,716 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,716 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,716 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,860 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,900 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,916 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,916 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,916 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,924 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,926 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,932 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,934 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,940 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,940 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,948 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,951 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,959 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a for region: 17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:19,959 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 17bc3062a59e914ed503fb224f492a7a Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:54:20,002 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:20,008 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:54:20,010 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:54:20,010 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:20,010 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:20,010 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:20,193 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,194 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:20,194 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:20,194 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:20,195 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:20,195 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:20,195 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:20,195 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:20,195 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:20,196 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:20,197 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 16:54:20,197 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:20,197 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:20,197 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:20,197 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:20,198 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:20,198 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:20,198 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:20,199 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:20,199 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:20,199 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:20,199 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:20,200 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,200 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:20,200 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:20,200 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:20,200 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,200 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,201 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,201 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:20,201 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:20,201 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:20,201 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:20,202 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:20,202 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961690202 2023-02-09 16:54:20,203 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:20,203 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,206 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:20,206 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:20,206 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:20,206 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961720206 2023-02-09 16:54:20,206 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961780206 2023-02-09 16:54:20,206 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:54:20,210 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:20,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:20,212 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:20,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:20,456 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:20,457 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:20,457 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:20,457 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:20,458 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:20,458 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:20,458 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:20,458 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:20,458 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:20,459 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:20,460 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 16:54:20,460 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:20,460 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:20,460 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:20,461 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:20,461 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:20,461 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:20,461 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:20,462 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:20,462 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:20,462 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:20,462 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:20,463 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,463 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN 2023-02-09 16:54:20,463 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:20,463 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:20,463 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,463 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,463 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,463 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=17bc3062a59e914ed503fb224f492a7a, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:20,463 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 17bc3062a59e914ed503fb224f492a7a, server=jenkins-hbase4.apache.org,43347,1675961647698 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:20,463 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:20,464 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:20,464 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:20,468 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961690468 2023-02-09 16:54:20,468 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:20,469 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,471 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:20,471 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=17bc3062a59e914ed503fb224f492a7a, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43347,1675961647698, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698, openSeqNum=2 2023-02-09 16:54:20,472 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:20,472 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961720472 2023-02-09 16:54:20,472 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961780472 2023-02-09 16:54:20,472 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:20,475 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961660475"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675961660475"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675961660475"}]},"ts":"1675961660475"} 2023-02-09 16:54:20,475 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961660475"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961660475"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961660475"}]},"ts":"1675961660475"} 2023-02-09 16:54:20,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961660475"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961660475"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961660475"}]},"ts":"1675961660475"} 2023-02-09 16:54:20,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, ASSIGN}] 2023-02-09 16:54:20,482 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, ASSIGN 2023-02-09 16:54:20,483 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:54:20,484 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, ASSIGN 2023-02-09 16:54:20,486 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:54:20,634 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:54:20,635 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0e0dc67407eec4115a0c4071ea62b937, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:20,635 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961660635"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961660635"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961660635"}]},"ts":"1675961660635"} 2023-02-09 16:54:20,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 0e0dc67407eec4115a0c4071ea62b937, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:20,638 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:54:20,641 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=3886eaed3353b979029f0f97c1c5b489, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:20,642 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961660641"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961660641"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961660641"}]},"ts":"1675961660641"} 2023-02-09 16:54:20,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 3886eaed3353b979029f0f97c1c5b489, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:20,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3886eaed3353b979029f0f97c1c5b489, NAME => 'testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 16:54:20,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:20,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:20,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,796 INFO [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,796 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1 2023-02-09 16:54:20,797 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1 2023-02-09 16:54:20,797 INFO [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3886eaed3353b979029f0f97c1c5b489 columnFamilyName cf1 2023-02-09 16:54:20,838 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a-bottom 2023-02-09 16:54:20,842 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb-bottom 2023-02-09 16:54:20,847 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4-bottom 2023-02-09 16:54:20,854 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904-bottom 2023-02-09 16:54:20,859 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be-bottom 2023-02-09 16:54:20,864 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7-bottom 2023-02-09 16:54:20,868 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c-bottom 2023-02-09 16:54:20,869 INFO [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(310): Store=3886eaed3353b979029f0f97c1c5b489/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:20,869 INFO [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,870 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2 2023-02-09 16:54:20,870 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2 2023-02-09 16:54:20,870 INFO [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3886eaed3353b979029f0f97c1c5b489 columnFamilyName cf2 2023-02-09 16:54:20,885 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026-bottom 2023-02-09 16:54:20,889 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41-bottom 2023-02-09 16:54:20,894 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04-bottom 2023-02-09 16:54:20,898 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2-bottom 2023-02-09 16:54:20,903 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78-bottom 2023-02-09 16:54:20,908 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d-bottom 2023-02-09 16:54:20,913 DEBUG [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e-bottom 2023-02-09 16:54:20,913 INFO [StoreOpener-3886eaed3353b979029f0f97c1c5b489-1] regionserver.HStore(310): Store=3886eaed3353b979029f0f97c1c5b489/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:20,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:20,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:20,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3886eaed3353b979029f0f97c1c5b489; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71194840, jitterRate=0.06088578701019287}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:20,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3886eaed3353b979029f0f97c1c5b489: Running coprocessor pre-open hook at 1675961660794Writing region info on filesystem at 1675961660794Initializing all the Stores at 1675961660795 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961660795Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961660795Cleaning up temporary data from old regions at 1675961660916 (+121 ms)Cleaning up detritus from prior splits at 1675961660916Running coprocessor post-open hooks at 1675961660919 (+3 ms)Region opened successfully at 1675961660919 2023-02-09 16:54:20,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489., pid=34, masterSystemTime=1675961660790 2023-02-09 16:54:20,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:20,921 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:20,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:20,921 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,921 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:20,922 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): 3886eaed3353b979029f0f97c1c5b489/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:20,922 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,922 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3886eaed3353b979029f0f97c1c5b489/cf1 in testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,922 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): 3886eaed3353b979029f0f97c1c5b489/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:20,922 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3886eaed3353b979029f0f97c1c5b489/cf2 in testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,922 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/.tmp, totalSize=34.9 K 2023-02-09 16:54:20,922 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/.tmp, totalSize=34.9 K 2023-02-09 16:54:20,923 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961656088 2023-02-09 16:54:20,923 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961656088 2023-02-09 16:54:20,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:20,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:20,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0e0dc67407eec4115a0c4071ea62b937, NAME => 'testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 16:54:20,924 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961656280 2023-02-09 16:54:20,924 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961656280 2023-02-09 16:54:20,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:20,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,924 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961656484 2023-02-09 16:54:20,925 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961656484 2023-02-09 16:54:20,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:20,925 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961656698 2023-02-09 16:54:20,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,925 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=3886eaed3353b979029f0f97c1c5b489, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:20,925 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961656698 2023-02-09 16:54:20,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,925 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961660925"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961660925"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961660925"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961660925"}]},"ts":"1675961660925"} 2023-02-09 16:54:20,925 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961656898 2023-02-09 16:54:20,926 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961656898 2023-02-09 16:54:20,926 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961657103 2023-02-09 16:54:20,927 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961657103 2023-02-09 16:54:20,927 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961657312 2023-02-09 16:54:20,927 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961657312 2023-02-09 16:54:20,928 INFO [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,930 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1 2023-02-09 16:54:20,930 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1 2023-02-09 16:54:20,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-09 16:54:20,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 3886eaed3353b979029f0f97c1c5b489, server=jenkins-hbase4.apache.org,43347,1675961647698 in 284 msec 2023-02-09 16:54:20,931 INFO [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e0dc67407eec4115a0c4071ea62b937 columnFamilyName cf1 2023-02-09 16:54:20,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, ASSIGN in 452 msec 2023-02-09 16:54:20,944 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3886eaed3353b979029f0f97c1c5b489#cf1#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:20,945 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): 3886eaed3353b979029f0f97c1c5b489#cf2#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:20,950 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a-top 2023-02-09 16:54:20,954 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb-top 2023-02-09 16:54:20,958 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4-top 2023-02-09 16:54:20,969 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904-top 2023-02-09 16:54:20,977 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be-top 2023-02-09 16:54:20,979 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/.tmp/cf1/7d56316990b048d990de0037700f713b as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/7d56316990b048d990de0037700f713b 2023-02-09 16:54:20,982 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7-top 2023-02-09 16:54:20,984 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/.tmp/cf2/a1076e5bea9c4a3bb0a21121bd6ba916 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/a1076e5bea9c4a3bb0a21121bd6ba916 2023-02-09 16:54:20,986 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3886eaed3353b979029f0f97c1c5b489/cf1 of 3886eaed3353b979029f0f97c1c5b489 into 7d56316990b048d990de0037700f713b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:20,986 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3886eaed3353b979029f0f97c1c5b489: 2023-02-09 16:54:20,987 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489., storeName=3886eaed3353b979029f0f97c1c5b489/cf1, priority=9, startTime=1675961660920; duration=0sec 2023-02-09 16:54:20,987 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:20,987 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c-top 2023-02-09 16:54:20,987 INFO [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(310): Store=0e0dc67407eec4115a0c4071ea62b937/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:20,988 INFO [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:20,989 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2 2023-02-09 16:54:20,989 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2 2023-02-09 16:54:20,989 INFO [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0e0dc67407eec4115a0c4071ea62b937 columnFamilyName cf2 2023-02-09 16:54:20,990 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3886eaed3353b979029f0f97c1c5b489/cf2 of 3886eaed3353b979029f0f97c1c5b489 into a1076e5bea9c4a3bb0a21121bd6ba916(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:20,990 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3886eaed3353b979029f0f97c1c5b489: 2023-02-09 16:54:20,990 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489., storeName=3886eaed3353b979029f0f97c1c5b489/cf2, priority=9, startTime=1675961660921; duration=0sec 2023-02-09 16:54:20,990 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:21,003 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026-top 2023-02-09 16:54:21,006 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41-top 2023-02-09 16:54:21,010 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04-top 2023-02-09 16:54:21,014 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2-top 2023-02-09 16:54:21,017 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78-top 2023-02-09 16:54:21,021 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d-top 2023-02-09 16:54:21,025 DEBUG [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e-top 2023-02-09 16:54:21,025 INFO [StoreOpener-0e0dc67407eec4115a0c4071ea62b937-1] regionserver.HStore(310): Store=0e0dc67407eec4115a0c4071ea62b937/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:21,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:21,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0e0dc67407eec4115a0c4071ea62b937; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69392182, jitterRate=0.03402408957481384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:21,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0e0dc67407eec4115a0c4071ea62b937: Running coprocessor pre-open hook at 1675961660925Writing region info on filesystem at 1675961660925Initializing all the Stores at 1675961660927 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961660927Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961660927Cleaning up temporary data from old regions at 1675961661027 (+100 ms)Cleaning up detritus from prior splits at 1675961661028 (+1 ms)Running coprocessor post-open hooks at 1675961661030 (+2 ms)Region opened successfully at 1675961661030 2023-02-09 16:54:21,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937., pid=33, masterSystemTime=1675961660790 2023-02-09 16:54:21,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:21,032 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:21,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:21,032 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:21,032 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,032 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): 0e0dc67407eec4115a0c4071ea62b937/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:21,032 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,032 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0e0dc67407eec4115a0c4071ea62b937/cf1 in testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,032 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): 0e0dc67407eec4115a0c4071ea62b937/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:21,032 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0e0dc67407eec4115a0c4071ea62b937/cf2 in testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,033 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7-top] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/.tmp, totalSize=34.9 K 2023-02-09 16:54:21,033 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04-top] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/.tmp, totalSize=34.9 K 2023-02-09 16:54:21,034 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961656088 2023-02-09 16:54:21,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,034 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961656088 2023-02-09 16:54:21,035 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961656280 2023-02-09 16:54:21,035 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0e0dc67407eec4115a0c4071ea62b937, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:21,035 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961661035"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961661035"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961661035"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961661035"}]},"ts":"1675961661035"} 2023-02-09 16:54:21,035 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961656280 2023-02-09 16:54:21,035 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961656484 2023-02-09 16:54:21,036 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961656484 2023-02-09 16:54:21,036 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961656698 2023-02-09 16:54:21,036 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675961656898 2023-02-09 16:54:21,037 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961656698 2023-02-09 16:54:21,037 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675961657103 2023-02-09 16:54:21,038 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675961656898 2023-02-09 16:54:21,038 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961657312 2023-02-09 16:54:21,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-09 16:54:21,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 0e0dc67407eec4115a0c4071ea62b937, server=jenkins-hbase4.apache.org,43347,1675961647698 in 400 msec 2023-02-09 16:54:21,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-09 16:54:21,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, ASSIGN in 561 msec 2023-02-09 16:54:21,044 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675961657103 2023-02-09 16:54:21,045 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961657312 2023-02-09 16:54:21,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 in 3.4690 sec 2023-02-09 16:54:21,061 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0e0dc67407eec4115a0c4071ea62b937#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:21,062 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): 0e0dc67407eec4115a0c4071ea62b937#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:21,079 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/.tmp/cf1/03cd71e38cc74121ae30035533a22b49 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/03cd71e38cc74121ae30035533a22b49 2023-02-09 16:54:21,080 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/.tmp/cf2/ca34485f65c34d7d8f23d0bce32688d9 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/ca34485f65c34d7d8f23d0bce32688d9 2023-02-09 16:54:21,084 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0e0dc67407eec4115a0c4071ea62b937/cf1 of 0e0dc67407eec4115a0c4071ea62b937 into 03cd71e38cc74121ae30035533a22b49(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:21,084 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0e0dc67407eec4115a0c4071ea62b937: 2023-02-09 16:54:21,084 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937., storeName=0e0dc67407eec4115a0c4071ea62b937/cf1, priority=9, startTime=1675961661031; duration=0sec 2023-02-09 16:54:21,085 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:21,086 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0e0dc67407eec4115a0c4071ea62b937/cf2 of 0e0dc67407eec4115a0c4071ea62b937 into ca34485f65c34d7d8f23d0bce32688d9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:21,086 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0e0dc67407eec4115a0c4071ea62b937: 2023-02-09 16:54:21,086 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937., storeName=0e0dc67407eec4115a0c4071ea62b937/cf2, priority=9, startTime=1675961661032; duration=0sec 2023-02-09 16:54:21,086 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:21,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43347] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:36868 deadline: 1675961721222, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a. is not online on jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:21,335 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:54:21,342 INFO [Listener at localhost/39015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 16:54:21,344 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:54:21,360 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:21,360 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,361 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,362 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,362 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,362 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-09 16:54:21,363 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-09 16:54:21,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,366 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,366 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 16:54:21,367 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961661367"}]},"ts":"1675961661367"} 2023-02-09 16:54:21,368 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-09 16:54:21,370 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-09 16:54:21,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, UNASSIGN}] 2023-02-09 16:54:21,372 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, UNASSIGN 2023-02-09 16:54:21,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=0e0dc67407eec4115a0c4071ea62b937, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:21,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961661374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961661374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961661374"}]},"ts":"1675961661374"} 2023-02-09 16:54:21,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 0e0dc67407eec4115a0c4071ea62b937, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:21,378 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, UNASSIGN 2023-02-09 16:54:21,378 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=3886eaed3353b979029f0f97c1c5b489, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:21,379 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961661378"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961661378"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961661378"}]},"ts":"1675961661378"} 2023-02-09 16:54:21,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 3886eaed3353b979029f0f97c1c5b489, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:21,467 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,467 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 16:54:21,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:21,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3886eaed3353b979029f0f97c1c5b489, disabling compactions & flushes 2023-02-09 16:54:21,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:21,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:21,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. after waiting 0 ms 2023-02-09 16:54:21,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:21,539 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7-bottom] to archive 2023-02-09 16:54:21,540 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:21,542 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,543 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,545 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,546 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,547 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,549 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,550 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,552 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04-bottom] to archive 2023-02-09 16:54:21,553 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:21,554 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,556 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,557 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,558 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,559 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,560 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,561 DEBUG [StoreCloser-testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:54:21,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:21,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489. 2023-02-09 16:54:21,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3886eaed3353b979029f0f97c1c5b489: Waiting for close lock at 1675961661527Running coprocessor pre-close hooks at 1675961661527Disabling compacts and flushes for region at 1675961661527Disabling writes for close at 1675961661527Writing region close event to WAL at 1675961661562 (+35 ms)Running coprocessor post-close hooks at 1675961661565 (+3 ms)Closed at 1675961661565 2023-02-09 16:54:21,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:21,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0e0dc67407eec4115a0c4071ea62b937, disabling compactions & flushes 2023-02-09 16:54:21,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. after waiting 0 ms 2023-02-09 16:54:21,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,567 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=3886eaed3353b979029f0f97c1c5b489, regionState=CLOSED 2023-02-09 16:54:21,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961661567"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961661567"}]},"ts":"1675961661567"} 2023-02-09 16:54:21,568 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7-top] to archive 2023-02-09 16:54:21,569 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:21,571 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-09 16:54:21,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 3886eaed3353b979029f0f97c1c5b489, server=jenkins-hbase4.apache.org,43347,1675961647698 in 189 msec 2023-02-09 16:54:21,573 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3886eaed3353b979029f0f97c1c5b489, UNASSIGN in 201 msec 2023-02-09 16:54:21,574 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,575 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,577 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,578 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,579 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,581 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04-top] to archive 2023-02-09 16:54:21,581 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:21,583 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,584 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,585 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,586 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,588 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,589 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,590 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:54:21,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:21,595 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937. 2023-02-09 16:54:21,595 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0e0dc67407eec4115a0c4071ea62b937: Waiting for close lock at 1675961661567Running coprocessor pre-close hooks at 1675961661567Disabling compacts and flushes for region at 1675961661567Disabling writes for close at 1675961661567Writing region close event to WAL at 1675961661591 (+24 ms)Running coprocessor post-close hooks at 1675961661595 (+4 ms)Closed at 1675961661595 2023-02-09 16:54:21,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,597 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=0e0dc67407eec4115a0c4071ea62b937, regionState=CLOSED 2023-02-09 16:54:21,598 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961661597"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961661597"}]},"ts":"1675961661597"} 2023-02-09 16:54:21,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-09 16:54:21,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 0e0dc67407eec4115a0c4071ea62b937, server=jenkins-hbase4.apache.org,43347,1675961647698 in 224 msec 2023-02-09 16:54:21,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-09 16:54:21,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=0e0dc67407eec4115a0c4071ea62b937, UNASSIGN in 230 msec 2023-02-09 16:54:21,605 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961661604"}]},"ts":"1675961661604"} 2023-02-09 16:54:21,606 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-09 16:54:21,607 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-09 16:54:21,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 245 msec 2023-02-09 16:54:21,669 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,669 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 16:54:21,670 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-09 16:54:21,670 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,670 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,671 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-09 16:54:21,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,673 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,674 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,674 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,674 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 16:54:21,680 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,680 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,680 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:21,683 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/recovered.edits] 2023-02-09 16:54:21,683 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/recovered.edits] 2023-02-09 16:54:21,684 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/recovered.edits] 2023-02-09 16:54:21,687 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/.regioninfo to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/.regioninfo 2023-02-09 16:54:21,687 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/1456800b4cb748fd83e27a0ee45cfd7a 2023-02-09 16:54:21,688 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/.regioninfo to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/.regioninfo 2023-02-09 16:54:21,689 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/229965d9e893483ea8a4d07f032d95cb 2023-02-09 16:54:21,691 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/4dfd42937a1d4e9a9e6c54892f222ce4 2023-02-09 16:54:21,691 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,691 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/1456800b4cb748fd83e27a0ee45cfd7a.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,692 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,693 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/63db2fd56776442d8bd7f50f55213904 2023-02-09 16:54:21,693 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/229965d9e893483ea8a4d07f032d95cb.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,694 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/9fc6980fd4d344f2abe403520675e1be 2023-02-09 16:54:21,694 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,695 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/4dfd42937a1d4e9a9e6c54892f222ce4.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,695 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/b54e4237840e41799c443137acffeec7 2023-02-09 16:54:21,696 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,696 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/63db2fd56776442d8bd7f50f55213904.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,697 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf1/f82f62afc5be4bd395f0ab516b66f05c 2023-02-09 16:54:21,697 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,698 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/9fc6980fd4d344f2abe403520675e1be.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,699 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,700 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/b54e4237840e41799c443137acffeec7.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,700 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/0edf2c81cb3d44da92f452c523b7d026 2023-02-09 16:54:21,700 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,701 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf1/f82f62afc5be4bd395f0ab516b66f05c.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,702 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/708d8023a2664abf80e51ebea3a74c41 2023-02-09 16:54:21,704 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/72ff67bf0da74cbe8b50b69e5eca0c04 2023-02-09 16:54:21,705 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,705 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/0edf2c81cb3d44da92f452c523b7d026.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,706 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/7f285d8c881948f28299fc457a9fb4f2 2023-02-09 16:54:21,706 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,707 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/708d8023a2664abf80e51ebea3a74c41.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,707 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/8ddb0f46753e437eafa8b3b79209cf78 2023-02-09 16:54:21,709 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,709 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/72ff67bf0da74cbe8b50b69e5eca0c04.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,709 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/abc6f97b082b4b829e4112ec98cd710d 2023-02-09 16:54:21,711 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,711 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/cf2/cee9b13fe78941cca81787dcdb88c45e 2023-02-09 16:54:21,711 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/7f285d8c881948f28299fc457a9fb4f2.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,713 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,713 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/8ddb0f46753e437eafa8b3b79209cf78.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,715 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/recovered.edits/85.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a/recovered.edits/85.seqid 2023-02-09 16:54:21,715 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,715 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/abc6f97b082b4b829e4112ec98cd710d.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,716 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,717 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/3886eaed3353b979029f0f97c1c5b489/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,717 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/0e0dc67407eec4115a0c4071ea62b937/cf2/cee9b13fe78941cca81787dcdb88c45e.17bc3062a59e914ed503fb224f492a7a 2023-02-09 16:54:21,720 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/7d56316990b048d990de0037700f713b to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf1/7d56316990b048d990de0037700f713b 2023-02-09 16:54:21,720 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/03cd71e38cc74121ae30035533a22b49 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf1/03cd71e38cc74121ae30035533a22b49 2023-02-09 16:54:21,722 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/a1076e5bea9c4a3bb0a21121bd6ba916 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/cf2/a1076e5bea9c4a3bb0a21121bd6ba916 2023-02-09 16:54:21,723 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/ca34485f65c34d7d8f23d0bce32688d9 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/cf2/ca34485f65c34d7d8f23d0bce32688d9 2023-02-09 16:54:21,725 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489/recovered.edits/90.seqid 2023-02-09 16:54:21,726 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937/recovered.edits/90.seqid 2023-02-09 16:54:21,726 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/3886eaed3353b979029f0f97c1c5b489 2023-02-09 16:54:21,726 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitWithoutPONR/0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:21,727 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 16:54:21,729 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,732 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-09 16:54:21,734 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-09 16:54:21,735 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,735 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-09 16:54:21,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961661735"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:21,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961661735"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:21,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961661735"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:21,737 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:54:21,737 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 17bc3062a59e914ed503fb224f492a7a, NAME => 'testSplitWithoutPONR,,1675961655582.17bc3062a59e914ed503fb224f492a7a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3886eaed3353b979029f0f97c1c5b489, NAME => 'testSplitWithoutPONR,,1675961657575.3886eaed3353b979029f0f97c1c5b489.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0e0dc67407eec4115a0c4071ea62b937, NAME => 'testSplitWithoutPONR,41,1675961657575.0e0dc67407eec4115a0c4071ea62b937.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 16:54:21,737 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-09 16:54:21,738 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961661737"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:21,739 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-09 16:54:21,741 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:21,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 70 msec 2023-02-09 16:54:21,775 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,775 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 16:54:21,776 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-09 16:54:21,792 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=398 (was 394) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:48412 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43814 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:48406 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43788 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43870 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x14051d1-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43850 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60128 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:44280 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43876 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60170 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60146 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:60158 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:48400 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43838 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:48402 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-755310874_15 at /127.0.0.1:43824 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=880 (was 827) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 223), ProcessCount=168 (was 168), AvailableMemoryMB=3282 (was 3455) 2023-02-09 16:54:21,806 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=398, OpenFileDescriptor=880, MaxFileDescriptor=60000, SystemLoadAverage=204, ProcessCount=168, AvailableMemoryMB=3281 2023-02-09 16:54:21,806 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:21,806 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:21,806 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:21,810 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:54:21,811 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:54:21,812 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:21,814 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:21,814 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:21,816 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:21,817 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae empty. 2023-02-09 16:54:21,817 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:21,817 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 16:54:21,827 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:21,828 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => c307239a1f0f6e7432449ec50648abae, NAME => 'testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:21,837 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:21,837 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing c307239a1f0f6e7432449ec50648abae, disabling compactions & flushes 2023-02-09 16:54:21,837 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:21,837 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:21,837 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. after waiting 0 ms 2023-02-09 16:54:21,837 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:21,837 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:21,837 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for c307239a1f0f6e7432449ec50648abae: Waiting for close lock at 1675961661837Disabling compacts and flushes for region at 1675961661837Disabling writes for close at 1675961661837Writing region close event to WAL at 1675961661837Closed at 1675961661837 2023-02-09 16:54:21,839 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:21,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961661840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961661840"}]},"ts":"1675961661840"} 2023-02-09 16:54:21,842 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:21,843 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:21,843 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961661843"}]},"ts":"1675961661843"} 2023-02-09 16:54:21,844 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 16:54:21,847 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:21,848 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:21,848 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:21,848 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:21,848 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:21,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN}] 2023-02-09 16:54:21,850 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:21,851 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37073,1675961647745; forceNewPlan=false, retain=false 2023-02-09 16:54:22,001 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:22,002 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,002 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961662002"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961662002"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961662002"}]},"ts":"1675961662002"} 2023-02-09 16:54:22,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:22,157 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,157 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 16:54:22,160 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51142, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 16:54:22,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c307239a1f0f6e7432449ec50648abae, NAME => 'testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:22,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:22,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:22,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,169 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,170 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1 2023-02-09 16:54:22,170 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1 2023-02-09 16:54:22,171 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c307239a1f0f6e7432449ec50648abae columnFamilyName cf1 2023-02-09 16:54:22,171 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(310): Store=c307239a1f0f6e7432449ec50648abae/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:22,172 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,173 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2 2023-02-09 16:54:22,173 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2 2023-02-09 16:54:22,173 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c307239a1f0f6e7432449ec50648abae columnFamilyName cf2 2023-02-09 16:54:22,174 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(310): Store=c307239a1f0f6e7432449ec50648abae/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:22,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:22,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:22,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:22,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c307239a1f0f6e7432449ec50648abae; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65176382, jitterRate=-0.028796225786209106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:22,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c307239a1f0f6e7432449ec50648abae: Running coprocessor pre-open hook at 1675961662168Writing region info on filesystem at 1675961662168Initializing all the Stores at 1675961662168Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961662169 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961662169Cleaning up temporary data from old regions at 1675961662176 (+7 ms)Cleaning up detritus from prior splits at 1675961662176Running coprocessor post-open hooks at 1675961662181 (+5 ms)Region opened successfully at 1675961662181 2023-02-09 16:54:22,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae., pid=43, masterSystemTime=1675961662157 2023-02-09 16:54:22,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,187 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961662186"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961662186"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961662186"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961662186"}]},"ts":"1675961662186"} 2023-02-09 16:54:22,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-09 16:54:22,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 in 184 msec 2023-02-09 16:54:22,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-09 16:54:22,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN in 342 msec 2023-02-09 16:54:22,193 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:22,193 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961662193"}]},"ts":"1675961662193"} 2023-02-09 16:54:22,195 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 16:54:22,204 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:22,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 393 msec 2023-02-09 16:54:22,315 DEBUG [Listener at localhost/39015] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 16:54:22,319 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51152, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 16:54:22,326 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,326 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,326 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:22,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 16:54:22,328 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,328 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,328 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 16:54:22,328 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:54:22,329 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,329 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,330 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,330 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,330 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,330 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,331 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,332 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,332 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,332 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,336 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,336 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:22,336 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,336 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. started... 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,337 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:54:22,337 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,337 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,337 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,337 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,339 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,350 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/c25796270dac4b27b6c8ec5f9b604887 2023-02-09 16:54:22,365 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/959178a3061b4e4eb0270c21df8b6426 2023-02-09 16:54:22,370 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/c25796270dac4b27b6c8ec5f9b604887 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 2023-02-09 16:54:22,374 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:22,375 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/959178a3061b4e4eb0270c21df8b6426 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 2023-02-09 16:54:22,379 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:22,380 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 43ms, sequenceid=6, compaction requested=false 2023-02-09 16:54:22,380 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,381 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:22,381 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,381 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:22,381 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:22,381 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,381 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,381 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,383 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,383 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,383 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,383 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 16:54:22,385 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:54:22,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a538536[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:22,385 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,386 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,386 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,386 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,387 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,387 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:22,387 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:22,388 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,388 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,388 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,389 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37073,1675961647745': 2023-02-09 16:54:22,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:54:22,392 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 16:54:22,392 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:22,392 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:22,392 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 16:54:22,392 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,396 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,396 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,399 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,399 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,402 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,402 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,402 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:22,402 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,403 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:22,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,404 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,404 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,404 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,503 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:22,503 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,504 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:22,519 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,520 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:22,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:22,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 16:54:22,522 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,522 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,522 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 16:54:22,522 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:54:22,523 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,523 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,524 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,525 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,526 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,527 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:22,527 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:22,527 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,527 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. started... 2023-02-09 16:54:22,527 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:22,528 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,529 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,529 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,529 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,529 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,529 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,537 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/076b1292dfa24f2ebc250e2cac3ac95f 2023-02-09 16:54:22,551 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/be1e425b515d467aaf695bccf8c4d684 2023-02-09 16:54:22,556 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/076b1292dfa24f2ebc250e2cac3ac95f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f 2023-02-09 16:54:22,561 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:22,562 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/be1e425b515d467aaf695bccf8c4d684 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 2023-02-09 16:54:22,566 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:22,567 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 40ms, sequenceid=19, compaction requested=false 2023-02-09 16:54:22,567 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:22,567 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,567 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:22,567 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,568 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,569 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,569 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,570 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,570 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 16:54:22,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c936f93[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:22,572 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:54:22,572 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,574 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,574 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:22,574 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:22,580 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,580 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,581 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,581 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37073,1675961647745': 2023-02-09 16:54:22,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:54:22,584 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 16:54:22,584 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:22,584 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:22,584 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 16:54:22,584 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,586 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,587 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,587 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,589 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,589 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,589 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,593 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,593 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:22,593 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:22,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:22,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,594 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,595 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,595 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,695 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:22,695 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,695 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:22,709 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,709 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,710 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:22,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:22,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 16:54:22,711 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 16:54:22,711 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:54:22,712 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,712 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,713 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,714 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,714 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,715 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,715 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,715 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,716 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:22,716 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:22,716 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,716 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. started... 2023-02-09 16:54:22,717 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:22,717 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,718 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,718 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,718 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,719 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,719 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,730 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/1ed08bd6f564492694a92752865e2192 2023-02-09 16:54:22,750 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/56f47c6011ed42ce875f105518ef58c4 2023-02-09 16:54:22,755 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/1ed08bd6f564492694a92752865e2192 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 2023-02-09 16:54:22,759 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:22,760 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/56f47c6011ed42ce875f105518ef58c4 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 2023-02-09 16:54:22,764 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:22,765 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 49ms, sequenceid=32, compaction requested=true 2023-02-09 16:54:22,765 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:22,765 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,765 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:22,766 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:22,766 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,766 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,766 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,769 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 16:54:22,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a36c679[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:22,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:54:22,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,773 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,773 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:22,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:22,773 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,774 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,774 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:22,775 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,775 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,775 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,775 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,777 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,777 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,777 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,777 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,777 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,777 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37073,1675961647745': 2023-02-09 16:54:22,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:54:22,780 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 16:54:22,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:22,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:22,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 16:54:22,780 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,782 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,783 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,783 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,784 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,785 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,785 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,786 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,791 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,791 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,791 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,791 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,792 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,892 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:22,892 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,892 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:22,908 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:22,908 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:22,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:22,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:22,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 16:54:22,910 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 16:54:22,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:54:22,911 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,911 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,912 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,912 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,912 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,912 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,912 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,913 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:22,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:22,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,914 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,914 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. started... 2023-02-09 16:54:22,915 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:22,915 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,917 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,917 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,917 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,917 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,917 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,925 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/a0e11cf7667a4968874f0df5a73a9863 2023-02-09 16:54:22,937 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/3a0b0834825048bb897d41eaca2eefb9 2023-02-09 16:54:22,942 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/a0e11cf7667a4968874f0df5a73a9863 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 2023-02-09 16:54:22,946 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:22,946 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/3a0b0834825048bb897d41eaca2eefb9 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 2023-02-09 16:54:22,950 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:22,951 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 37ms, sequenceid=45, compaction requested=true 2023-02-09 16:54:22,951 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:22,951 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:22,951 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:22,952 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:22,952 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,952 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:22,952 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:22,953 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,953 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,953 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,953 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:22,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 16:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68c2b4cc[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:22,955 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:54:22,955 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,956 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,956 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,956 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,957 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:22,957 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:22,957 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:22,958 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,958 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,958 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,959 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:22,959 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,959 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,959 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,959 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,959 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,959 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37073,1675961647745': 2023-02-09 16:54:22,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:54:22,962 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 16:54:22,962 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:22,962 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:22,962 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 16:54:22,962 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:22,964 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,964 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,964 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:22,964 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,965 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,966 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:22,967 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:22,968 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,968 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,970 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 16:54:22,970 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:22,970 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:22,970 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:22,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:22,971 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:22,972 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,972 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:22,972 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:22,972 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:22,972 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,071 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:23,071 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:23,071 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:23,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:23,085 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:23,086 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:23,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:23,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:23,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 16:54:23,088 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 16:54:23,088 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,088 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 16:54:23,088 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:54:23,088 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,088 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,090 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,090 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,090 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,090 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,091 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:23,091 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:23,092 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:23,092 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:23,092 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,092 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:23,092 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:23,092 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:23,092 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:23,092 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. started... 2023-02-09 16:54:23,093 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:23,094 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,094 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,094 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,094 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,094 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:23,094 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:23,112 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/2e56ac01c1784d158dd24f9317f8e99e 2023-02-09 16:54:23,124 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/1e9ced27178643ceab7038303407622a 2023-02-09 16:54:23,128 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/2e56ac01c1784d158dd24f9317f8e99e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e 2023-02-09 16:54:23,132 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:23,133 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/1e9ced27178643ceab7038303407622a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a 2023-02-09 16:54:23,136 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:23,137 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 44ms, sequenceid=58, compaction requested=true 2023-02-09 16:54:23,137 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:23,137 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:23,137 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:23,138 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:23,138 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,138 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:23,138 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:23,141 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,141 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:23,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:23,141 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,141 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:23,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:23,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:23,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:23,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 16:54:23,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eeef6d7[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:23,143 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:54:23,143 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,144 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,144 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:23,144 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:23,145 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:23,154 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,154 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:23,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:23,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37073,1675961647745': 2023-02-09 16:54:23,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:54:23,158 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 16:54:23,158 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:23,158 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:23,158 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 16:54:23,158 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:23,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,159 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:23,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,161 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,162 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,163 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:23,163 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,167 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 16:54:23,167 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:23,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,167 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,167 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:23,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:23,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,168 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,169 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,269 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:23,269 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:23,269 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:23,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:23,284 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:23,284 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:23,285 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:23,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:23,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 16:54:23,287 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 16:54:23,287 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,287 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 16:54:23,287 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:54:23,287 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,287 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,288 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,288 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,288 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,288 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,289 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,289 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:23,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:23,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:23,290 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:23,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:23,291 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 16:54:23,291 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:23,291 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. started... 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:23,291 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:23,293 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,293 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,293 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,293 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:23,294 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,294 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:23,306 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/e94aa385960b4b20a5c2ccd7bc233e4f 2023-02-09 16:54:23,319 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/f2edd476d76a4e1c894e5b1f5050d12c 2023-02-09 16:54:23,324 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/e94aa385960b4b20a5c2ccd7bc233e4f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f 2023-02-09 16:54:23,328 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:23,329 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/f2edd476d76a4e1c894e5b1f5050d12c as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c 2023-02-09 16:54:23,333 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:23,334 INFO [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 43ms, sequenceid=71, compaction requested=true 2023-02-09 16:54:23,334 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:23,334 DEBUG [rs(jenkins-hbase4.apache.org,37073,1675961647745)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:23,334 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:23,334 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:23,334 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,334 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 16:54:23,334 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 16:54:23,336 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,336 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:23,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:23,337 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,337 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:23,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:23,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:23,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:23,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 16:54:23,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2996e96e[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:23,338 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:54:23,338 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:23,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:23,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:23,340 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 16:54:23,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:23,341 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:23,341 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,341 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:23,342 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,342 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:23,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:23,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:23,342 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 16:54:23,342 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,342 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,342 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:23,342 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 16:54:23,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:23,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:23,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:23,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37073,1675961647745': 2023-02-09 16:54:23,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:54:23,345 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 16:54:23,345 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:23,345 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:23,345 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 16:54:23,345 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:23,348 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,348 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,348 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,348 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,349 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:23,348 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,349 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,349 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:23,349 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,350 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:23,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,351 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:23,352 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 16:54:23,352 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,359 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:23,359 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 16:54:23,359 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:23,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:23,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:23,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:23,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,359 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 16:54:23,360 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 16:54:23,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:23,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:23,460 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:23,460 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:23,460 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:23,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:23,721 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 16:54:23,722 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:23,723 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,726 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:23,726 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:23,726 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:23,974 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,974 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:23,974 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:23,974 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:23,974 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-09 16:54:23,974 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:23,974 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:23,975 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:23,975 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:23,975 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:23,976 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 16:54:23,977 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:23,977 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:23,977 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:23,977 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:23,978 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:23,978 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:23,978 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:23,978 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:23,979 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:23,979 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:23,979 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:23,979 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:23,980 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:23,980 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,980 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:23,980 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,980 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,980 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,980 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:23,981 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:23,982 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:23,984 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961693984 2023-02-09 16:54:23,985 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:23,985 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,988 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:23,988 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:23,988 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:23,988 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961723988 2023-02-09 16:54:23,988 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961783988 2023-02-09 16:54:23,988 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:23,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:23,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:23,990 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:23,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:23,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:24,239 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,239 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:24,239 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:24,239 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:24,241 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:24,241 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:24,241 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:24,241 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:24,242 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:24,242 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:24,243 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 16:54:24,244 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:24,244 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:24,244 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:24,244 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:24,245 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,245 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:24,245 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,245 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,246 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,246 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:24,246 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,246 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,247 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:24,247 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,247 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:24,247 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,247 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,247 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,247 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:24,248 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:24,248 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:24,254 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961694254 2023-02-09 16:54:24,256 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,256 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:24,258 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:24,259 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:24,259 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:24,259 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961724259 2023-02-09 16:54:24,259 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961784259 2023-02-09 16:54:24,259 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:24,260 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:24,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN}] 2023-02-09 16:54:24,262 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:24,262 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:24,262 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:24,510 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:24,510 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:24,511 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:24,512 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 16:54:24,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:24,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:24,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:24,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:24,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:24,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:24,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:24,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,516 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:24,517 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,517 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,517 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,517 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:24,517 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:24,518 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:24,518 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961694518 2023-02-09 16:54:24,519 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:24,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,522 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:24,522 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:24,522 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:24,522 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961724522 2023-02-09 16:54:24,522 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961784522 2023-02-09 16:54:24,523 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:24,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN}] 2023-02-09 16:54:24,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:24,525 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:24,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:24,526 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961664526"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961664526"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961664526"}]},"ts":"1675961664526"} 2023-02-09 16:54:24,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:24,528 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:24,528 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:24,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:24,528 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:24,773 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,773 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:24,773 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:24,773 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:24,774 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:24,774 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:24,774 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:24,774 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:24,775 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:24,775 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:24,776 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:24,776 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 16:54:24,776 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:24,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:24,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:24,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:24,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:24,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:24,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:24,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:24,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:24,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:24,780 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:24,780 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,780 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,780 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:24,780 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:24,780 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:24,780 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:24,781 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:24,781 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:24,781 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:24,782 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961694782 2023-02-09 16:54:24,782 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:24,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:24,785 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:24,785 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:24,785 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:24,785 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961724785 2023-02-09 16:54:24,785 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961784785 2023-02-09 16:54:24,785 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:24,786 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:24,786 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961664786"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961664786"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961664786"}]},"ts":"1675961664786"} 2023-02-09 16:54:24,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:24,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:24,940 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:24,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c307239a1f0f6e7432449ec50648abae, disabling compactions & flushes 2023-02-09 16:54:24,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:24,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:24,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. after waiting 0 ms 2023-02-09 16:54:24,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:24,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing c307239a1f0f6e7432449ec50648abae 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:54:24,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/57efc5ab768643bd9e9dfd0ae0a060cc 2023-02-09 16:54:24,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/a864b1b54e18471d820d4f5e2675270d 2023-02-09 16:54:24,977 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/57efc5ab768643bd9e9dfd0ae0a060cc as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc 2023-02-09 16:54:24,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:54:24,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/a864b1b54e18471d820d4f5e2675270d as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d 2023-02-09 16:54:24,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:54:24,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 47ms, sequenceid=82, compaction requested=true 2023-02-09 16:54:24,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:54:24,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:24,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:24,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c307239a1f0f6e7432449ec50648abae: Waiting for close lock at 1675961664941Running coprocessor pre-close hooks at 1675961664941Disabling compacts and flushes for region at 1675961664941Disabling writes for close at 1675961664941Obtaining lock to block concurrent updates at 1675961664941Preparing flush snapshotting stores in c307239a1f0f6e7432449ec50648abae at 1675961664941Finished memstore snapshotting testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675961664941Flushing stores of testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. at 1675961664942 (+1 ms)Flushing c307239a1f0f6e7432449ec50648abae/cf1: creating writer at 1675961664943 (+1 ms)Flushing c307239a1f0f6e7432449ec50648abae/cf1: appending metadata at 1675961664945 (+2 ms)Flushing c307239a1f0f6e7432449ec50648abae/cf1: closing flushed file at 1675961664945Flushing c307239a1f0f6e7432449ec50648abae/cf2: creating writer at 1675961664957 (+12 ms)Flushing c307239a1f0f6e7432449ec50648abae/cf2: appending metadata at 1675961664959 (+2 ms)Flushing c307239a1f0f6e7432449ec50648abae/cf2: closing flushed file at 1675961664959Flushing c307239a1f0f6e7432449ec50648abae/cf1: reopening flushed file at 1675961664978 (+19 ms)Flushing c307239a1f0f6e7432449ec50648abae/cf2: reopening flushed file at 1675961664983 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for c307239a1f0f6e7432449ec50648abae in 47ms, sequenceid=82, compaction requested=true at 1675961664988 (+5 ms)Writing region close event to WAL at 1675961664990 (+2 ms)Running coprocessor post-close hooks at 1675961664993 (+3 ms)Closed at 1675961664993 2023-02-09 16:54:24,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:24,996 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED 2023-02-09 16:54:24,996 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961664996"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961664996"}]},"ts":"1675961664996"} 2023-02-09 16:54:24,997 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:24,997 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:24,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:24,998 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:25,036 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:25,036 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:25,037 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:25,038 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:25,038 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 16:54:25,038 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 16:54:25,038 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:25,039 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:25,039 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:25,039 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:25,039 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,039 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:25,040 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,040 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,040 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,040 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:25,040 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:25,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:25,042 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:25,042 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,042 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,042 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,042 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:25,042 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,043 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:25,043 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:25,043 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:25,044 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:25,044 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:25,044 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961695044 2023-02-09 16:54:25,044 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:25,045 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:25,047 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:25,047 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:25,048 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:25,048 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961725048 2023-02-09 16:54:25,048 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961785048 2023-02-09 16:54:25,048 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:25,048 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:25,048 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961665048"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961665048"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961665048"}]},"ts":"1675961665048"} 2023-02-09 16:54:25,049 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:25,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 16:54:25,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 in 262 msec 2023-02-09 16:54:25,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:25,052 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:25,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:25,298 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,298 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:25,298 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:25,298 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:25,300 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:25,300 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:25,300 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:25,300 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:25,300 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:25,301 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:25,301 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:25,302 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 16:54:25,302 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:25,302 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:25,303 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:25,303 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:25,303 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:25,303 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,303 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:25,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:25,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,305 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:25,305 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,306 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,306 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:25,306 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:25,306 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,306 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,306 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,306 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:25,306 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:25,306 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:25,307 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:25,307 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:25,307 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:25,307 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961695307 2023-02-09 16:54:25,307 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:25,308 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,310 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:25,311 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:25,311 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:25,311 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961725311 2023-02-09 16:54:25,311 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961785311 2023-02-09 16:54:25,311 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:25,311 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,313 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:25,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 16:54:25,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN in 790 msec 2023-02-09 16:54:25,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:25,319 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:25,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:25,561 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,561 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:25,561 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:25,561 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:25,562 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:25,562 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:25,562 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:25,562 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:25,562 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:25,563 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:25,563 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:25,563 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:25,564 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:25,564 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:25,564 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:25,564 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:25,564 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:25,565 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,565 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:25,565 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,565 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:25,566 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,566 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:25,566 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,567 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:25,567 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:25,568 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,568 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:25,568 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:25,568 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:25,568 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:25,568 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:25,568 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:25,569 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:25,569 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:25,572 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961695572 2023-02-09 16:54:25,573 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,573 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:25,575 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:25,576 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:25,576 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:25,576 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961725576 2023-02-09 16:54:25,576 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961785576 2023-02-09 16:54:25,576 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:54:25,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:25,590 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=c307239a1f0f6e7432449ec50648abae, threads=14 2023-02-09 16:54:25,591 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,591 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,591 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,591 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,591 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,591 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,592 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,592 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,592 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,593 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,593 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,593 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,593 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,593 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,762 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,790 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,790 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,791 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,792 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,800 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,803 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,805 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,806 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,808 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,813 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,813 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,818 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,818 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:25,818 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region c307239a1f0f6e7432449ec50648abae Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:54:25,852 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:25,852 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:25,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:25,852 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:26,076 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:26,077 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:26,077 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:26,078 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:26,079 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 16:54:26,079 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:26,079 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:26,080 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:26,080 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:26,080 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:26,080 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:26,081 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:26,081 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:26,081 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:26,081 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:26,081 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:26,082 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:26,082 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:26,082 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:26,082 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:26,083 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,083 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:26,083 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:26,083 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:26,083 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,083 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,083 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,083 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:26,083 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:26,083 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:26,084 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:26,084 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:26,085 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961696085 2023-02-09 16:54:26,086 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:26,086 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,089 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:26,089 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:26,089 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:26,089 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961726089 2023-02-09 16:54:26,089 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961786089 2023-02-09 16:54:26,089 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:26,098 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=c307239a1f0f6e7432449ec50648abae, threads=14 2023-02-09 16:54:26,099 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,099 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,099 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,099 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,099 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,103 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,103 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,103 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,104 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,104 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,104 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,104 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,104 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,106 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,246 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,246 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,260 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,296 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:54:26,297 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,297 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,301 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,302 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,302 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,308 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,308 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,309 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,324 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,333 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,719 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f for region: c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:26,720 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region c307239a1f0f6e7432449ec50648abae Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:54:26,763 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:26,768 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/5dbef3147b010b12df9eb8c5e71c12c8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:54:26,770 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/614b0c169020f96ce7313ce87ffc96ce/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:54:26,770 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:26,770 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:26,770 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:26,839 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,840 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:26,840 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:26,840 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:26,841 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:26,841 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:26,841 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:26,841 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:26,841 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:26,842 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:26,842 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 16:54:26,842 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:26,842 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:26,843 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:26,843 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:26,843 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:26,843 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:26,844 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:26,844 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:26,844 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:26,844 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:26,844 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:26,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:26,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:26,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:26,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:26,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:26,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:26,848 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:26,848 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,848 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,848 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,848 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:26,848 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:26,848 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 16:54:26,849 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:26,849 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:26,849 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961696849 2023-02-09 16:54:26,849 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:26,850 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,852 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:26,853 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:26,853 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:26,853 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961726853 2023-02-09 16:54:26,853 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961786853 2023-02-09 16:54:26,853 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:26,856 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:26,859 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:26,859 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:26,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:26,859 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:27,104 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,104 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:27,104 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:27,104 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:27,105 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:27,105 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:27,105 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:27,105 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:27,105 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:27,106 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:27,107 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 16:54:27,107 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:27,107 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:27,107 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:27,108 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:27,108 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:27,108 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:27,108 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,109 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:27,109 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,109 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,109 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,109 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:27,109 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:27,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:27,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:27,110 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:27,111 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,111 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,111 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,111 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,111 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,111 DEBUG [Listener at localhost/39015] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,111 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:27,112 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:27,112 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:27,112 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961697112 2023-02-09 16:54:27,113 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:27,115 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,117 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:27,117 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:27,117 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:27,117 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961727117 2023-02-09 16:54:27,118 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961787118 2023-02-09 16:54:27,118 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:54:27,118 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:27,130 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:27,130 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:27,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:27,131 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:27,369 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,369 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:27,369 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:27,369 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:27,370 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:27,370 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:27,370 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:27,370 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:27,370 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:27,371 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:27,372 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 16:54:27,372 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:27,372 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:27,372 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:27,373 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:27,373 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:27,373 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:27,373 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,373 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:27,373 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:27,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:27,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:27,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:27,376 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:27,376 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,376 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,376 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,376 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,376 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,376 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:27,377 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:27,377 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:27,378 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961697378 2023-02-09 16:54:27,380 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:27,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,383 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:27,383 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:27,383 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:27,383 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961727383 2023-02-09 16:54:27,383 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961787383 2023-02-09 16:54:27,383 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:54:27,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:27,385 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:27,385 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:27,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:27,386 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:27,634 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,634 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:27,634 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:27,634 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:27,635 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:27,635 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:27,635 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:27,635 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:27,635 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:27,636 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:27,637 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 16:54:27,637 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:27,637 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:27,637 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:27,638 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:27,638 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:27,638 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:27,639 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,639 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:27,639 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,639 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,640 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,640 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:27,640 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,640 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,641 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:27,641 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,641 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:27,642 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:27,642 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:27,642 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,642 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,642 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,642 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,642 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,642 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 16:54:27,643 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:27,643 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:27,643 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961697643 2023-02-09 16:54:27,643 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:27,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,647 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:27,647 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:27,647 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:27,647 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961727647 2023-02-09 16:54:27,647 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961787647 2023-02-09 16:54:27,647 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:27,647 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:27,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:27,649 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:27,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:27,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:27,897 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,897 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:27,898 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:27,898 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:27,900 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:27,900 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:27,900 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:27,900 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:27,900 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:27,901 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:27,902 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 16:54:27,902 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:27,902 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:27,902 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:27,902 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:27,903 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:27,903 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:27,903 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,903 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:27,903 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,903 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:27,904 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,904 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:27,904 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,904 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:27,904 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:27,905 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,905 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:27,905 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:27,905 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:27,905 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,905 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,905 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,905 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,905 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:27,905 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:27,906 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:27,907 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:27,907 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961697907 2023-02-09 16:54:27,908 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:27,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:27,911 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:27,911 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:27,911 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:27,911 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961727911 2023-02-09 16:54:27,911 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961787911 2023-02-09 16:54:27,911 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:27,911 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:27,915 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN] 2023-02-09 16:54:27,915 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:27,915 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:27,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:27,915 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:28,161 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,161 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:28,161 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:28,161 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:28,162 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:28,162 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:28,162 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:28,162 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:28,162 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:28,163 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:28,163 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 16:54:28,163 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:54:28,163 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:54:28,163 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 16:54:28,164 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:28,164 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:28,164 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:28,164 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:28,164 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,165 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:28,165 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,165 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,165 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,165 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:28,166 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,166 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,166 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:28,167 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,167 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,167 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:28,167 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:28,168 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:28,168 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:28,168 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:28,169 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:28,169 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:28,172 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961698172 2023-02-09 16:54:28,172 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:28,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,175 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,176 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:28,176 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:28,177 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:28,177 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961728177 2023-02-09 16:54:28,177 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961788177 2023-02-09 16:54:28,177 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:54:28,177 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:28,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:28,182 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:28,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:28,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:28,427 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,427 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:28,427 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:28,427 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:28,427 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:28,427 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:28,428 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:28,428 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:28,428 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:28,428 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:28,429 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:54:28,429 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 16:54:28,429 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:28,430 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:28,430 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:28,430 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:28,430 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,431 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:28,431 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,431 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,431 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,431 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:28,431 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,432 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,432 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:28,432 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,432 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,432 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:28,432 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,432 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,432 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,433 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:28,433 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:28,433 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:28,434 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:28,434 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:28,435 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961698435 2023-02-09 16:54:28,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,437 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:28,438 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,440 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:28,440 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:28,441 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:28,441 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961728441 2023-02-09 16:54:28,441 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961788441 2023-02-09 16:54:28,441 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:28,441 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:28,444 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:28,444 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:28,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:28,444 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:28,691 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,691 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:28,691 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:28,691 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:28,691 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:28,692 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:28,692 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:28,692 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:28,692 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:28,692 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:28,693 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 16:54:28,693 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 16:54:28,693 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:28,694 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:28,694 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:28,694 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:28,694 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,695 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:28,695 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,695 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,696 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,696 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:28,696 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,696 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,697 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:28,697 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,697 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,697 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:28,697 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,698 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,698 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,698 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:28,698 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 16:54:28,699 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:28,699 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:28,699 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:28,700 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961698700 2023-02-09 16:54:28,701 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:28,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,703 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,704 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:28,705 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:28,705 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:28,705 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961728705 2023-02-09 16:54:28,705 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961788705 2023-02-09 16:54:28,705 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:54:28,705 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:28,707 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:28,707 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 16:54:28,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:28,707 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:28,955 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,955 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:28,955 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:28,955 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:28,955 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:28,956 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:28,956 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:28,956 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:28,956 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:28,956 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:28,957 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 16:54:28,957 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 16:54:28,957 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:28,958 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:28,958 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:28,958 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:28,958 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,959 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:28,959 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,959 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:28,959 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,959 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:28,960 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,960 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:28,960 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:28,960 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,960 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,960 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:28,961 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,961 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,961 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,961 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:54:28,961 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:28,961 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:28,962 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:28,962 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:28,962 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961698962 2023-02-09 16:54:28,962 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:28,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:28,967 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,968 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:28,969 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:28,969 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:28,969 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961728969 2023-02-09 16:54:28,969 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961788969 2023-02-09 16:54:28,969 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-09 16:54:28,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:54:28,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce exec-time=5.2460 sec 2023-02-09 16:54:28,972 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:28,972 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,37073,1675961647745; forceNewPlan=false, retain=true 2023-02-09 16:54:29,123 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:29,124 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:29,124 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961669124"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961669124"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961669124"}]},"ts":"1675961669124"} 2023-02-09 16:54:29,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:29,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:54:29,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:29,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:54:29,127 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:54:29,219 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:29,220 WARN [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:54:29,220 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:54:29,220 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:54:29,221 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:54:29,222 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-09 16:54:29,222 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:54:29,223 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:54:29,223 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:54:29,223 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:54:29,223 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:29,223 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:54:29,224 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:29,224 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:54:29,224 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:29,224 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:54:29,224 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:29,224 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:54:29,225 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,225 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:54:29,225 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:29,225 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:54:29,225 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:29,225 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 16:54:29,226 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:54:29,226 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:54:29,226 INFO [Listener at localhost/39015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae to restore RIT 2023-02-09 16:54:29,227 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961699227 2023-02-09 16:54:29,228 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:54:29,228 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:54:29,230 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:54:29,230 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=c307239a1f0f6e7432449ec50648abae, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,37073,1675961647745, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745, openSeqNum=2 2023-02-09 16:54:29,230 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:54:29,230 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961729230 2023-02-09 16:54:29,230 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961789230 2023-02-09 16:54:29,230 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:54:29,230 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:29,231 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961669230"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961669230"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961669230"}]},"ts":"1675961669230"} 2023-02-09 16:54:29,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:29,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c307239a1f0f6e7432449ec50648abae, NAME => 'testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:29,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:29,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:29,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,390 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,391 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1 2023-02-09 16:54:29,391 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1 2023-02-09 16:54:29,391 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c307239a1f0f6e7432449ec50648abae columnFamilyName cf1 2023-02-09 16:54:29,398 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f 2023-02-09 16:54:29,403 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 2023-02-09 16:54:29,406 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e 2023-02-09 16:54:29,409 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc 2023-02-09 16:54:29,418 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 2023-02-09 16:54:29,421 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 2023-02-09 16:54:29,425 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f 2023-02-09 16:54:29,426 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(310): Store=c307239a1f0f6e7432449ec50648abae/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:29,426 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,426 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2 2023-02-09 16:54:29,427 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2 2023-02-09 16:54:29,427 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c307239a1f0f6e7432449ec50648abae columnFamilyName cf2 2023-02-09 16:54:29,436 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a 2023-02-09 16:54:29,442 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 2023-02-09 16:54:29,450 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 2023-02-09 16:54:29,454 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 2023-02-09 16:54:29,458 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d 2023-02-09 16:54:29,465 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 2023-02-09 16:54:29,469 DEBUG [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c 2023-02-09 16:54:29,469 INFO [StoreOpener-c307239a1f0f6e7432449ec50648abae-1] regionserver.HStore(310): Store=c307239a1f0f6e7432449ec50648abae/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:29,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.splits 2023-02-09 16:54:29,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:29,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c307239a1f0f6e7432449ec50648abae; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60218764, jitterRate=-0.10267049074172974}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:29,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c307239a1f0f6e7432449ec50648abae: Running coprocessor pre-open hook at 1675961669389Writing region info on filesystem at 1675961669389Initializing all the Stores at 1675961669389Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961669390 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961669390Cleaning up temporary data from old regions at 1675961669471 (+81 ms)Cleaning up detritus from prior splits at 1675961669472 (+1 ms)Running coprocessor post-open hooks at 1675961669475 (+3 ms)Region opened successfully at 1675961669475 2023-02-09 16:54:29,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae., pid=48, masterSystemTime=1675961669384 2023-02-09 16:54:29,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:29,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:29,478 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:29,478 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:29,479 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 16:54:29,479 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 16:54:29,480 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.HStore(1912): c307239a1f0f6e7432449ec50648abae/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:29,480 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.HStore(1912): c307239a1f0f6e7432449ec50648abae/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:29,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,480 INFO [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c307239a1f0f6e7432449ec50648abae/cf1 in testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,480 INFO [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c307239a1f0f6e7432449ec50648abae/cf2 in testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,480 INFO [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp, totalSize=34.9 K 2023-02-09 16:54:29,480 INFO [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp, totalSize=34.9 K 2023-02-09 16:54:29,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:29,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961669480"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961669480"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961669480"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961669480"}]},"ts":"1675961669480"} 2023-02-09 16:54:29,480 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting c25796270dac4b27b6c8ec5f9b604887, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961662320 2023-02-09 16:54:29,481 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting 959178a3061b4e4eb0270c21df8b6426, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961662320 2023-02-09 16:54:29,481 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting 076b1292dfa24f2ebc250e2cac3ac95f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961662505 2023-02-09 16:54:29,481 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting be1e425b515d467aaf695bccf8c4d684, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961662505 2023-02-09 16:54:29,481 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting 1ed08bd6f564492694a92752865e2192, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961662696 2023-02-09 16:54:29,481 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-09 16:54:29,482 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting 56f47c6011ed42ce875f105518ef58c4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961662696 2023-02-09 16:54:29,482 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting 3a0b0834825048bb897d41eaca2eefb9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961662894 2023-02-09 16:54:29,482 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting a0e11cf7667a4968874f0df5a73a9863, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961662894 2023-02-09 16:54:29,483 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting 1e9ced27178643ceab7038303407622a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961663073 2023-02-09 16:54:29,483 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting 2e56ac01c1784d158dd24f9317f8e99e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961663073 2023-02-09 16:54:29,484 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting f2edd476d76a4e1c894e5b1f5050d12c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961663271 2023-02-09 16:54:29,484 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting e94aa385960b4b20a5c2ccd7bc233e4f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961663271 2023-02-09 16:54:29,484 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] compactions.Compactor(207): Compacting a864b1b54e18471d820d4f5e2675270d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961663462 2023-02-09 16:54:29,484 DEBUG [Listener at localhost/39015] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-09 16:54:29,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-09 16:54:29,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 in 250 msec 2023-02-09 16:54:29,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN in 1.5710 sec 2023-02-09 16:54:29,486 INFO [Listener at localhost/39015] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 16:54:29,487 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] compactions.Compactor(207): Compacting 57efc5ab768643bd9e9dfd0ae0a060cc, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961663462 2023-02-09 16:54:29,490 INFO [Listener at localhost/39015] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-09 16:54:29,490 INFO [Listener at localhost/39015] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 16:54:29,490 INFO [Listener at localhost/39015] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-09 16:54:29,513 INFO [RS:1;jenkins-hbase4:37073-shortCompactions-0] throttle.PressureAwareThroughputController(145): c307239a1f0f6e7432449ec50648abae#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:29,513 INFO [RS:1;jenkins-hbase4:37073-longCompactions-0] throttle.PressureAwareThroughputController(145): c307239a1f0f6e7432449ec50648abae#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:29,538 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf1/30f0c15a0ff045a98397fbe2767b46a7 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/30f0c15a0ff045a98397fbe2767b46a7 2023-02-09 16:54:29,544 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/.tmp/cf2/7265f5fc4cbd42209bda2745f214b79e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/7265f5fc4cbd42209bda2745f214b79e 2023-02-09 16:54:29,549 INFO [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c307239a1f0f6e7432449ec50648abae/cf1 of c307239a1f0f6e7432449ec50648abae into 30f0c15a0ff045a98397fbe2767b46a7(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:29,549 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:29,549 INFO [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae., storeName=c307239a1f0f6e7432449ec50648abae/cf1, priority=9, startTime=1675961669476; duration=0sec 2023-02-09 16:54:29,550 DEBUG [RS:1;jenkins-hbase4:37073-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:29,552 INFO [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c307239a1f0f6e7432449ec50648abae/cf2 of c307239a1f0f6e7432449ec50648abae into 7265f5fc4cbd42209bda2745f214b79e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:29,553 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c307239a1f0f6e7432449ec50648abae: 2023-02-09 16:54:29,553 INFO [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae., storeName=c307239a1f0f6e7432449ec50648abae/cf2, priority=9, startTime=1675961669477; duration=0sec 2023-02-09 16:54:29,553 DEBUG [RS:1;jenkins-hbase4:37073-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:29,562 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:29,562 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,562 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,564 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,564 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,565 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-09 16:54:29,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-09 16:54:29,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,568 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,568 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 16:54:29,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961669569"}]},"ts":"1675961669569"} 2023-02-09 16:54:29,571 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 16:54:29,572 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-09 16:54:29,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN}] 2023-02-09 16:54:29,575 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN 2023-02-09 16:54:29,576 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:29,576 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961669576"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961669576"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961669576"}]},"ts":"1675961669576"} 2023-02-09 16:54:29,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:29,670 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,670 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 16:54:29,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c307239a1f0f6e7432449ec50648abae, disabling compactions & flushes 2023-02-09 16:54:29,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. after waiting 0 ms 2023-02-09 16:54:29,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,731 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc] to archive 2023-02-09 16:54:29,732 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:29,734 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/c25796270dac4b27b6c8ec5f9b604887 2023-02-09 16:54:29,735 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/076b1292dfa24f2ebc250e2cac3ac95f 2023-02-09 16:54:29,736 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/1ed08bd6f564492694a92752865e2192 2023-02-09 16:54:29,737 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/a0e11cf7667a4968874f0df5a73a9863 2023-02-09 16:54:29,738 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/2e56ac01c1784d158dd24f9317f8e99e 2023-02-09 16:54:29,739 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/e94aa385960b4b20a5c2ccd7bc233e4f 2023-02-09 16:54:29,741 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/57efc5ab768643bd9e9dfd0ae0a060cc 2023-02-09 16:54:29,745 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d] to archive 2023-02-09 16:54:29,746 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:29,747 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/959178a3061b4e4eb0270c21df8b6426 2023-02-09 16:54:29,749 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/be1e425b515d467aaf695bccf8c4d684 2023-02-09 16:54:29,750 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/56f47c6011ed42ce875f105518ef58c4 2023-02-09 16:54:29,751 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/3a0b0834825048bb897d41eaca2eefb9 2023-02-09 16:54:29,752 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/1e9ced27178643ceab7038303407622a 2023-02-09 16:54:29,753 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/f2edd476d76a4e1c894e5b1f5050d12c 2023-02-09 16:54:29,754 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/a864b1b54e18471d820d4f5e2675270d 2023-02-09 16:54:29,757 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:54:29,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:29,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae. 2023-02-09 16:54:29,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c307239a1f0f6e7432449ec50648abae: Waiting for close lock at 1675961669730Running coprocessor pre-close hooks at 1675961669730Disabling compacts and flushes for region at 1675961669730Disabling writes for close at 1675961669730Writing region close event to WAL at 1675961669755 (+25 ms)Running coprocessor post-close hooks at 1675961669758 (+3 ms)Closed at 1675961669758 2023-02-09 16:54:29,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,760 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=c307239a1f0f6e7432449ec50648abae, regionState=CLOSED 2023-02-09 16:54:29,760 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961669760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961669760"}]},"ts":"1675961669760"} 2023-02-09 16:54:29,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 16:54:29,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure c307239a1f0f6e7432449ec50648abae, server=jenkins-hbase4.apache.org,37073,1675961647745 in 183 msec 2023-02-09 16:54:29,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 16:54:29,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, UNASSIGN in 190 msec 2023-02-09 16:54:29,765 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961669765"}]},"ts":"1675961669765"} 2023-02-09 16:54:29,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 16:54:29,768 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-09 16:54:29,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 203 msec 2023-02-09 16:54:29,871 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,871 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 16:54:29,872 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-09 16:54:29,872 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,872 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,873 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-09 16:54:29,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,875 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,876 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,876 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,876 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 16:54:29,879 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,881 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/recovered.edits] 2023-02-09 16:54:29,883 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/30f0c15a0ff045a98397fbe2767b46a7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf1/30f0c15a0ff045a98397fbe2767b46a7 2023-02-09 16:54:29,885 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/7265f5fc4cbd42209bda2745f214b79e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/cf2/7265f5fc4cbd42209bda2745f214b79e 2023-02-09 16:54:29,887 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae/recovered.edits/90.seqid 2023-02-09 16:54:29,888 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackAndDoubleExecution/c307239a1f0f6e7432449ec50648abae 2023-02-09 16:54:29,888 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 16:54:29,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,892 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-09 16:54:29,894 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-09 16:54:29,895 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,895 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-09 16:54:29,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961669895"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:29,897 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 16:54:29,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c307239a1f0f6e7432449ec50648abae, NAME => 'testRollbackAndDoubleExecution,,1675961661811.c307239a1f0f6e7432449ec50648abae.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 16:54:29,897 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-09 16:54:29,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961669897"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:29,898 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-09 16:54:29,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:54:29,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 28 msec 2023-02-09 16:54:29,977 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:29,977 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:29,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 16:54:29,978 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-09 16:54:29,993 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402 (was 398) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39428 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:51410 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39422 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:51436 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39976 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39958 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:37073-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39456 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:51458 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39426 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:51414 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39998 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:51430 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:51450 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39982 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39962 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1286408939_15 at /127.0.0.1:39440 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=856 (was 880), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 204), ProcessCount=168 (was 168), AvailableMemoryMB=3186 (was 3281) 2023-02-09 16:54:30,007 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402, OpenFileDescriptor=856, MaxFileDescriptor=60000, SystemLoadAverage=197, ProcessCount=168, AvailableMemoryMB=3185 2023-02-09 16:54:30,007 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:30,007 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:30,007 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:30,011 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:54:30,012 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:54:30,013 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:30,014 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:30,015 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:30,016 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,017 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd empty. 2023-02-09 16:54:30,017 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 16:54:30,027 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:30,028 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2c16f3ae420a962237b43fe9bcb426dd, NAME => 'testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:30,047 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:30,047 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 2c16f3ae420a962237b43fe9bcb426dd, disabling compactions & flushes 2023-02-09 16:54:30,047 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,047 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,047 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. after waiting 0 ms 2023-02-09 16:54:30,047 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,047 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,047 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 2c16f3ae420a962237b43fe9bcb426dd: Waiting for close lock at 1675961670047Disabling compacts and flushes for region at 1675961670047Disabling writes for close at 1675961670047Writing region close event to WAL at 1675961670047Closed at 1675961670047 2023-02-09 16:54:30,049 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:30,050 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675961670050"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961670050"}]},"ts":"1675961670050"} 2023-02-09 16:54:30,052 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:30,053 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:30,053 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961670053"}]},"ts":"1675961670053"} 2023-02-09 16:54:30,054 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-09 16:54:30,056 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:30,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:30,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:30,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:30,057 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:30,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, ASSIGN}] 2023-02-09 16:54:30,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, ASSIGN 2023-02-09 16:54:30,060 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:30,210 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:30,211 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=2c16f3ae420a962237b43fe9bcb426dd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:30,211 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675961670211"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961670211"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961670211"}]},"ts":"1675961670211"} 2023-02-09 16:54:30,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 2c16f3ae420a962237b43fe9bcb426dd, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:30,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2c16f3ae420a962237b43fe9bcb426dd, NAME => 'testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:30,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:30,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:30,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,372 INFO [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,374 DEBUG [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/cf1 2023-02-09 16:54:30,374 DEBUG [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/cf1 2023-02-09 16:54:30,374 INFO [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c16f3ae420a962237b43fe9bcb426dd columnFamilyName cf1 2023-02-09 16:54:30,375 INFO [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] regionserver.HStore(310): Store=2c16f3ae420a962237b43fe9bcb426dd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:30,375 INFO [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,376 DEBUG [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/cf2 2023-02-09 16:54:30,376 DEBUG [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/cf2 2023-02-09 16:54:30,376 INFO [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c16f3ae420a962237b43fe9bcb426dd columnFamilyName cf2 2023-02-09 16:54:30,377 INFO [StoreOpener-2c16f3ae420a962237b43fe9bcb426dd-1] regionserver.HStore(310): Store=2c16f3ae420a962237b43fe9bcb426dd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:30,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:30,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:30,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2c16f3ae420a962237b43fe9bcb426dd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64674645, jitterRate=-0.03627268970012665}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:30,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2c16f3ae420a962237b43fe9bcb426dd: Running coprocessor pre-open hook at 1675961670372Writing region info on filesystem at 1675961670372Initializing all the Stores at 1675961670372Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961670372Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961670372Cleaning up temporary data from old regions at 1675961670379 (+7 ms)Cleaning up detritus from prior splits at 1675961670379Running coprocessor post-open hooks at 1675961670383 (+4 ms)Region opened successfully at 1675961670383 2023-02-09 16:54:30,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd., pid=55, masterSystemTime=1675961670364 2023-02-09 16:54:30,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,386 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=2c16f3ae420a962237b43fe9bcb426dd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:30,386 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675961670386"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961670386"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961670386"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961670386"}]},"ts":"1675961670386"} 2023-02-09 16:54:30,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-09 16:54:30,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 2c16f3ae420a962237b43fe9bcb426dd, server=jenkins-hbase4.apache.org,33383,1675961647775 in 174 msec 2023-02-09 16:54:30,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-09 16:54:30,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, ASSIGN in 331 msec 2023-02-09 16:54:30,391 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:30,391 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961670391"}]},"ts":"1675961670391"} 2023-02-09 16:54:30,392 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-09 16:54:30,395 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:30,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 383 msec 2023-02-09 16:54:30,514 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:30,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:30,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:30,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:30,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:30,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, UNASSIGN}] 2023-02-09 16:54:30,519 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, UNASSIGN 2023-02-09 16:54:30,520 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=2c16f3ae420a962237b43fe9bcb426dd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:30,520 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675961670519"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961670519"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961670519"}]},"ts":"1675961670519"} 2023-02-09 16:54:30,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 2c16f3ae420a962237b43fe9bcb426dd, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:30,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2c16f3ae420a962237b43fe9bcb426dd, disabling compactions & flushes 2023-02-09 16:54:30,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. after waiting 0 ms 2023-02-09 16:54:30,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 16:54:30,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:30,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd. 2023-02-09 16:54:30,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2c16f3ae420a962237b43fe9bcb426dd: Waiting for close lock at 1675961670673Running coprocessor pre-close hooks at 1675961670673Disabling compacts and flushes for region at 1675961670673Disabling writes for close at 1675961670674 (+1 ms)Writing region close event to WAL at 1675961670675 (+1 ms)Running coprocessor post-close hooks at 1675961670678 (+3 ms)Closed at 1675961670678 2023-02-09 16:54:30,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:30,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=2c16f3ae420a962237b43fe9bcb426dd, regionState=CLOSED 2023-02-09 16:54:30,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675961670680"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961670680"}]},"ts":"1675961670680"} 2023-02-09 16:54:30,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-09 16:54:30,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 2c16f3ae420a962237b43fe9bcb426dd, server=jenkins-hbase4.apache.org,33383,1675961647775 in 160 msec 2023-02-09 16:54:30,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 16:54:30,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c16f3ae420a962237b43fe9bcb426dd, UNASSIGN in 165 msec 2023-02-09 16:54:30,707 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 16:54:30,709 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 16:54:30,710 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675961670710"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675961670710"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675961670710"}]},"ts":"1675961670710"} 2023-02-09 16:54:30,710 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961670710"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961670710"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961670710"}]},"ts":"1675961670710"} 2023-02-09 16:54:30,711 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961670710"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961670710"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961670710"}]},"ts":"1675961670710"} 2023-02-09 16:54:30,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, ASSIGN}] 2023-02-09 16:54:30,714 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, ASSIGN 2023-02-09 16:54:30,715 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:30,715 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, ASSIGN 2023-02-09 16:54:30,716 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:30,865 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:54:30,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=8b180db08753a36ce5add8496d915265, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:30,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961670867"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961670867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961670867"}]},"ts":"1675961670867"} 2023-02-09 16:54:30,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 8b180db08753a36ce5add8496d915265, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:30,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=ac7cbf4002f2c7ee631b20295b01672c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:30,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961670871"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961670871"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961670871"}]},"ts":"1675961670871"} 2023-02-09 16:54:30,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure ac7cbf4002f2c7ee631b20295b01672c, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:31,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ac7cbf4002f2c7ee631b20295b01672c, NAME => 'testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 16:54:31,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:31,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:31,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,025 INFO [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,027 DEBUG [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/cf1 2023-02-09 16:54:31,027 DEBUG [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/cf1 2023-02-09 16:54:31,027 INFO [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ac7cbf4002f2c7ee631b20295b01672c columnFamilyName cf1 2023-02-09 16:54:31,028 INFO [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] regionserver.HStore(310): Store=ac7cbf4002f2c7ee631b20295b01672c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:31,028 INFO [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,029 DEBUG [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/cf2 2023-02-09 16:54:31,029 DEBUG [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/cf2 2023-02-09 16:54:31,029 INFO [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ac7cbf4002f2c7ee631b20295b01672c columnFamilyName cf2 2023-02-09 16:54:31,030 INFO [StoreOpener-ac7cbf4002f2c7ee631b20295b01672c-1] regionserver.HStore(310): Store=ac7cbf4002f2c7ee631b20295b01672c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:31,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:31,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ac7cbf4002f2c7ee631b20295b01672c; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62050473, jitterRate=-0.07537589967250824}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:31,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ac7cbf4002f2c7ee631b20295b01672c: Running coprocessor pre-open hook at 1675961671024Writing region info on filesystem at 1675961671024Initializing all the Stores at 1675961671025 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961671025Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961671025Cleaning up temporary data from old regions at 1675961671032 (+7 ms)Cleaning up detritus from prior splits at 1675961671032Running coprocessor post-open hooks at 1675961671034 (+2 ms)Region opened successfully at 1675961671034 2023-02-09 16:54:31,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c., pid=62, masterSystemTime=1675961671020 2023-02-09 16:54:31,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8b180db08753a36ce5add8496d915265, NAME => 'testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 16:54:31,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=ac7cbf4002f2c7ee631b20295b01672c, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:31,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:31,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,037 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961671037"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961671037"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961671037"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961671037"}]},"ts":"1675961671037"} 2023-02-09 16:54:31,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:31,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,039 INFO [StoreOpener-8b180db08753a36ce5add8496d915265-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,040 DEBUG [StoreOpener-8b180db08753a36ce5add8496d915265-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/cf1 2023-02-09 16:54:31,040 DEBUG [StoreOpener-8b180db08753a36ce5add8496d915265-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/cf1 2023-02-09 16:54:31,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-09 16:54:31,041 INFO [StoreOpener-8b180db08753a36ce5add8496d915265-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b180db08753a36ce5add8496d915265 columnFamilyName cf1 2023-02-09 16:54:31,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure ac7cbf4002f2c7ee631b20295b01672c, server=jenkins-hbase4.apache.org,33383,1675961647775 in 166 msec 2023-02-09 16:54:31,041 INFO [StoreOpener-8b180db08753a36ce5add8496d915265-1] regionserver.HStore(310): Store=8b180db08753a36ce5add8496d915265/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:31,041 INFO [StoreOpener-8b180db08753a36ce5add8496d915265-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, ASSIGN in 327 msec 2023-02-09 16:54:31,042 DEBUG [StoreOpener-8b180db08753a36ce5add8496d915265-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/cf2 2023-02-09 16:54:31,043 DEBUG [StoreOpener-8b180db08753a36ce5add8496d915265-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/cf2 2023-02-09 16:54:31,043 INFO [StoreOpener-8b180db08753a36ce5add8496d915265-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b180db08753a36ce5add8496d915265 columnFamilyName cf2 2023-02-09 16:54:31,043 INFO [StoreOpener-8b180db08753a36ce5add8496d915265-1] regionserver.HStore(310): Store=8b180db08753a36ce5add8496d915265/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:31,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:31,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8b180db08753a36ce5add8496d915265; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75299387, jitterRate=0.12204830348491669}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:31,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8b180db08753a36ce5add8496d915265: Running coprocessor pre-open hook at 1675961671038Writing region info on filesystem at 1675961671038Initializing all the Stores at 1675961671038Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961671038Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961671039 (+1 ms)Cleaning up temporary data from old regions at 1675961671046 (+7 ms)Cleaning up detritus from prior splits at 1675961671046Running coprocessor post-open hooks at 1675961671049 (+3 ms)Region opened successfully at 1675961671050 (+1 ms) 2023-02-09 16:54:31,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265., pid=61, masterSystemTime=1675961671020 2023-02-09 16:54:31,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=8b180db08753a36ce5add8496d915265, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:31,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961671052"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961671052"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961671052"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961671052"}]},"ts":"1675961671052"} 2023-02-09 16:54:31,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-09 16:54:31,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 8b180db08753a36ce5add8496d915265, server=jenkins-hbase4.apache.org,33383,1675961647775 in 184 msec 2023-02-09 16:54:31,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-09 16:54:31,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, ASSIGN in 341 msec 2023-02-09 16:54:31,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c in 542 msec 2023-02-09 16:54:31,268 DEBUG [hconnection-0x82ecaf6-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 16:54:31,273 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56108, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 16:54:31,274 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:54:31,277 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:31,277 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,277 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,278 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,279 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,279 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,279 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,282 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,282 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 16:54:31,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961671282"}]},"ts":"1675961671282"} 2023-02-09 16:54:31,284 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-09 16:54:31,286 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-09 16:54:31,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, UNASSIGN}] 2023-02-09 16:54:31,287 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, UNASSIGN 2023-02-09 16:54:31,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=ac7cbf4002f2c7ee631b20295b01672c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:31,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961671287"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961671287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961671287"}]},"ts":"1675961671287"} 2023-02-09 16:54:31,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure ac7cbf4002f2c7ee631b20295b01672c, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:31,290 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, UNASSIGN 2023-02-09 16:54:31,291 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=8b180db08753a36ce5add8496d915265, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:31,291 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961671291"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961671291"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961671291"}]},"ts":"1675961671291"} 2023-02-09 16:54:31,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 8b180db08753a36ce5add8496d915265, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:31,383 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,383 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 16:54:31,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ac7cbf4002f2c7ee631b20295b01672c, disabling compactions & flushes 2023-02-09 16:54:31,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. after waiting 0 ms 2023-02-09 16:54:31,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 16:54:31,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:31,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c. 2023-02-09 16:54:31,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ac7cbf4002f2c7ee631b20295b01672c: Waiting for close lock at 1675961671440Running coprocessor pre-close hooks at 1675961671440Disabling compacts and flushes for region at 1675961671440Disabling writes for close at 1675961671441 (+1 ms)Writing region close event to WAL at 1675961671442 (+1 ms)Running coprocessor post-close hooks at 1675961671446 (+4 ms)Closed at 1675961671446 2023-02-09 16:54:31,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8b180db08753a36ce5add8496d915265, disabling compactions & flushes 2023-02-09 16:54:31,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,448 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. after waiting 0 ms 2023-02-09 16:54:31,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=ac7cbf4002f2c7ee631b20295b01672c, regionState=CLOSED 2023-02-09 16:54:31,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961671449"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961671449"}]},"ts":"1675961671449"} 2023-02-09 16:54:31,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 16:54:31,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure ac7cbf4002f2c7ee631b20295b01672c, server=jenkins-hbase4.apache.org,33383,1675961647775 in 162 msec 2023-02-09 16:54:31,452 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 16:54:31,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:31,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265. 2023-02-09 16:54:31,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8b180db08753a36ce5add8496d915265: Waiting for close lock at 1675961671447Running coprocessor pre-close hooks at 1675961671447Disabling compacts and flushes for region at 1675961671447Disabling writes for close at 1675961671449 (+2 ms)Writing region close event to WAL at 1675961671449Running coprocessor post-close hooks at 1675961671453 (+4 ms)Closed at 1675961671453 2023-02-09 16:54:31,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ac7cbf4002f2c7ee631b20295b01672c, UNASSIGN in 166 msec 2023-02-09 16:54:31,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,455 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=8b180db08753a36ce5add8496d915265, regionState=CLOSED 2023-02-09 16:54:31,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961671455"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961671455"}]},"ts":"1675961671455"} 2023-02-09 16:54:31,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-09 16:54:31,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 8b180db08753a36ce5add8496d915265, server=jenkins-hbase4.apache.org,33383,1675961647775 in 164 msec 2023-02-09 16:54:31,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-09 16:54:31,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=8b180db08753a36ce5add8496d915265, UNASSIGN in 171 msec 2023-02-09 16:54:31,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961671459"}]},"ts":"1675961671459"} 2023-02-09 16:54:31,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-09 16:54:31,462 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-09 16:54:31,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 184 msec 2023-02-09 16:54:31,584 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,584 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 16:54:31,585 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-09 16:54:31,585 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,585 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,585 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,588 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,588 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,588 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 16:54:31,593 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:31,593 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,593 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,595 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/recovered.edits] 2023-02-09 16:54:31,595 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/recovered.edits] 2023-02-09 16:54:31,595 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/recovered.edits] 2023-02-09 16:54:31,601 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/recovered.edits/7.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265/recovered.edits/7.seqid 2023-02-09 16:54:31,601 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/recovered.edits/7.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c/recovered.edits/7.seqid 2023-02-09 16:54:31,601 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/8b180db08753a36ce5add8496d915265 2023-02-09 16:54:31,602 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/recovered.edits/4.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd/recovered.edits/4.seqid 2023-02-09 16:54:31,602 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:54:31,602 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c16f3ae420a962237b43fe9bcb426dd 2023-02-09 16:54:31,602 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 16:54:31,604 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,606 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-09 16:54:31,607 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-09 16:54:31,608 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,608 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-09 16:54:31,608 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961671608"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:31,608 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961671608"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:31,608 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961671608"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:31,610 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:54:31,610 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2c16f3ae420a962237b43fe9bcb426dd, NAME => 'testSplitTableRegionNoStoreFile,,1675961670012.2c16f3ae420a962237b43fe9bcb426dd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8b180db08753a36ce5add8496d915265, NAME => 'testSplitTableRegionNoStoreFile,,1675961670514.8b180db08753a36ce5add8496d915265.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => ac7cbf4002f2c7ee631b20295b01672c, NAME => 'testSplitTableRegionNoStoreFile,41,1675961670514.ac7cbf4002f2c7ee631b20295b01672c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 16:54:31,610 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-09 16:54:31,610 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961671610"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:31,611 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-09 16:54:31,613 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:54:31,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 28 msec 2023-02-09 16:54:31,689 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,689 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 16:54:31,690 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-09 16:54:31,705 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=409 (was 402) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_903158096_15 at /127.0.0.1:39422 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_903158096_15 at /127.0.0.1:39958 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=858 (was 856) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=181 (was 197), ProcessCount=168 (was 168), AvailableMemoryMB=3184 (was 3185) 2023-02-09 16:54:31,719 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=409, OpenFileDescriptor=858, MaxFileDescriptor=60000, SystemLoadAverage=181, ProcessCount=168, AvailableMemoryMB=3184 2023-02-09 16:54:31,719 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:31,719 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:31,719 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:31,721 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:54:31,722 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:54:31,723 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:31,724 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:31,725 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:31,726 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:31,727 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a empty. 2023-02-09 16:54:31,727 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:31,727 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 16:54:31,740 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:31,741 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7e8ba877d307230da0baa92e22cac26a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:31,748 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:31,748 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 7e8ba877d307230da0baa92e22cac26a, disabling compactions & flushes 2023-02-09 16:54:31,748 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:31,748 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:31,749 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. after waiting 0 ms 2023-02-09 16:54:31,749 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:31,749 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:31,749 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 7e8ba877d307230da0baa92e22cac26a: Waiting for close lock at 1675961671748Disabling compacts and flushes for region at 1675961671748Disabling writes for close at 1675961671749 (+1 ms)Writing region close event to WAL at 1675961671749Closed at 1675961671749 2023-02-09 16:54:31,751 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:31,751 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675961671751"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961671751"}]},"ts":"1675961671751"} 2023-02-09 16:54:31,753 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:31,753 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:31,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961671754"}]},"ts":"1675961671754"} 2023-02-09 16:54:31,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-09 16:54:31,759 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:31,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:31,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:31,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:31,760 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:31,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, ASSIGN}] 2023-02-09 16:54:31,762 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, ASSIGN 2023-02-09 16:54:31,762 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:31,913 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:31,914 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=7e8ba877d307230da0baa92e22cac26a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:31,914 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675961671914"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961671914"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961671914"}]},"ts":"1675961671914"} 2023-02-09 16:54:31,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 7e8ba877d307230da0baa92e22cac26a, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:32,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e8ba877d307230da0baa92e22cac26a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:32,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:32,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:32,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,072 INFO [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,073 DEBUG [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1 2023-02-09 16:54:32,073 DEBUG [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1 2023-02-09 16:54:32,074 INFO [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e8ba877d307230da0baa92e22cac26a columnFamilyName cf1 2023-02-09 16:54:32,074 INFO [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] regionserver.HStore(310): Store=7e8ba877d307230da0baa92e22cac26a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:32,074 INFO [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,076 DEBUG [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2 2023-02-09 16:54:32,076 DEBUG [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2 2023-02-09 16:54:32,076 INFO [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e8ba877d307230da0baa92e22cac26a columnFamilyName cf2 2023-02-09 16:54:32,077 INFO [StoreOpener-7e8ba877d307230da0baa92e22cac26a-1] regionserver.HStore(310): Store=7e8ba877d307230da0baa92e22cac26a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:32,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:32,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:32,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7e8ba877d307230da0baa92e22cac26a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72592720, jitterRate=0.08171582221984863}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:32,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7e8ba877d307230da0baa92e22cac26a: Running coprocessor pre-open hook at 1675961672071Writing region info on filesystem at 1675961672071Initializing all the Stores at 1675961672071Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961672071Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961672072 (+1 ms)Cleaning up temporary data from old regions at 1675961672079 (+7 ms)Cleaning up detritus from prior splits at 1675961672079Running coprocessor post-open hooks at 1675961672084 (+5 ms)Region opened successfully at 1675961672084 2023-02-09 16:54:32,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a., pid=71, masterSystemTime=1675961672067 2023-02-09 16:54:32,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,087 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=7e8ba877d307230da0baa92e22cac26a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675961672087"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961672087"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961672087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961672087"}]},"ts":"1675961672087"} 2023-02-09 16:54:32,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-09 16:54:32,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 7e8ba877d307230da0baa92e22cac26a, server=jenkins-hbase4.apache.org,33383,1675961647775 in 174 msec 2023-02-09 16:54:32,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-09 16:54:32,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, ASSIGN in 331 msec 2023-02-09 16:54:32,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:32,094 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961672094"}]},"ts":"1675961672094"} 2023-02-09 16:54:32,095 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-09 16:54:32,097 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:32,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 375 msec 2023-02-09 16:54:32,231 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,231 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:32,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,234 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,234 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,234 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:32,234 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:32,234 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,234 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,236 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,236 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,236 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,236 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,237 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,237 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,238 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,238 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,238 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,239 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,239 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:32,239 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:32,240 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:54:32,241 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,242 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,242 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,242 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,242 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,242 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,254 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/ffd216f8f03c43acaddfb74ba2128f68 2023-02-09 16:54:32,268 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/09437cf1d1a3492eac56f901331af0e0 2023-02-09 16:54:32,272 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/ffd216f8f03c43acaddfb74ba2128f68 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68 2023-02-09 16:54:32,276 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:32,276 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/09437cf1d1a3492eac56f901331af0e0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0 2023-02-09 16:54:32,280 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:54:32,281 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 41ms, sequenceid=6, compaction requested=false 2023-02-09 16:54:32,281 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,281 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:32,281 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,282 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:32,282 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:32,282 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,282 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,282 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,283 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@358ff72b[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:32,285 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:32,286 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,288 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,288 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,288 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,290 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,290 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,291 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,291 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:32,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:32,294 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:32,294 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:32,294 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:32,294 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,294 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,296 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,297 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,297 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,299 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,299 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,299 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,306 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:32,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,306 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:32,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,307 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,407 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:32,408 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,408 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:32,422 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,422 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,423 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:32,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:32,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,427 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,427 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,427 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:32,427 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:32,428 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,428 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,429 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,429 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,429 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,429 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,430 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,430 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,431 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,431 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,431 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,431 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,431 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,432 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,432 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,432 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,432 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:32,432 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:32,433 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,434 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,434 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,435 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,435 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,435 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,442 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/b5adafa95567426fb5667dd81cd2fbf1 2023-02-09 16:54:32,454 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/3ba98099972245b69922170361b38d08 2023-02-09 16:54:32,458 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/b5adafa95567426fb5667dd81cd2fbf1 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1 2023-02-09 16:54:32,462 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:32,463 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/3ba98099972245b69922170361b38d08 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08 2023-02-09 16:54:32,467 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:54:32,467 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 35ms, sequenceid=19, compaction requested=false 2023-02-09 16:54:32,468 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:32,468 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,468 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:32,468 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:32,468 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,468 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,468 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,470 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,470 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,471 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,471 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:32,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48965182[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:32,473 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:32,473 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,474 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:32,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,474 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,475 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:32,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,475 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:32,476 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,476 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,476 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,477 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,477 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,477 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,479 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,479 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,479 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,479 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:32,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:32,482 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:32,482 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:32,482 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:32,482 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,482 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,484 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,486 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,487 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,487 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,488 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,489 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,492 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,492 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:32,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,492 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,492 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,493 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,593 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:32,593 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,593 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:32,605 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,605 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,606 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:32,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:32,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,608 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,608 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,608 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:32,608 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:32,608 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,608 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,609 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,609 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,609 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,609 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,610 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,610 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:32,612 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:32,612 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,611 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,612 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:32,612 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,612 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,612 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,612 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,612 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:32,614 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,614 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,614 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,614 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,614 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,614 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,622 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/84423ee5535c4a86b9e26dc58df7c83b 2023-02-09 16:54:32,633 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/11b980d0a89d4af6a21d9d02cec64300 2023-02-09 16:54:32,637 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/84423ee5535c4a86b9e26dc58df7c83b as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b 2023-02-09 16:54:32,641 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:32,641 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/11b980d0a89d4af6a21d9d02cec64300 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300 2023-02-09 16:54:32,646 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:54:32,647 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 35ms, sequenceid=32, compaction requested=true 2023-02-09 16:54:32,647 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:32,647 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,647 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:32,647 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:32,647 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,647 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,647 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,653 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,653 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,654 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,654 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:32,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1875e09[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:32,656 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:32,656 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,658 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,658 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:32,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,658 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:32,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:32,659 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:32,660 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,660 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,660 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,660 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,662 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,662 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,662 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,663 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,663 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,663 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:32,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:32,666 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:32,666 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:32,667 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:32,667 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,667 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,669 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,669 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,670 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,672 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,672 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,673 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,675 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,675 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:32,675 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,675 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,676 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,776 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:32,776 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,776 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:32,791 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,792 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,792 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:32,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:32,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,794 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:32,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:32,795 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,795 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,796 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,796 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,796 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,796 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,797 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,797 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,797 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,798 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,798 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:32,800 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,800 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,800 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,800 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:32,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,800 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:32,800 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,801 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,801 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,801 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,803 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,804 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,806 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,806 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,812 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/9afad62c8d9f473bbe0180074ce2336e 2023-02-09 16:54:32,826 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/4f22fb1a9f194088ad314974946111b7 2023-02-09 16:54:32,831 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/9afad62c8d9f473bbe0180074ce2336e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e 2023-02-09 16:54:32,836 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:32,837 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/4f22fb1a9f194088ad314974946111b7 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7 2023-02-09 16:54:32,840 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:54:32,841 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 41ms, sequenceid=45, compaction requested=true 2023-02-09 16:54:32,841 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:32,841 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,841 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:32,841 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:32,841 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,841 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,841 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,844 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,844 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,845 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,845 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:32,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f440ace[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:32,847 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:32,847 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,848 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:32,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:32,848 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:32,849 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:32,850 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,850 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,850 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,850 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,850 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,850 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,851 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,851 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:32,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:32,854 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:32,854 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:32,854 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:32,854 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,854 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:32,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,859 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:32,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,860 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,860 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:32,861 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:32,862 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,862 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,863 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,865 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,865 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,865 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:32,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:32,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,866 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:32,866 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:32,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:32,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,966 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:32,966 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,966 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:32,980 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:32,980 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:32,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:32,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:32,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,982 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:32,982 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,982 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:32,982 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:32,982 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,982 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,983 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,983 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,983 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:32,983 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:32,984 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:32,984 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,985 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,985 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,985 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:32,985 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,985 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,986 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:32,986 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:32,986 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:32,986 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:32,986 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:32,990 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,991 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,991 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,991 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,991 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:32,991 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:32,996 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/c15e2085e3114933b9082753f22bc248 2023-02-09 16:54:33,011 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 2023-02-09 16:54:33,015 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/c15e2085e3114933b9082753f22bc248 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248 2023-02-09 16:54:33,020 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:33,020 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 2023-02-09 16:54:33,024 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:54:33,025 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 39ms, sequenceid=58, compaction requested=true 2023-02-09 16:54:33,025 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:33,025 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,025 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:33,025 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:33,025 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,025 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,025 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,027 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,027 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,027 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,027 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:33,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55b880a2[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:33,029 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:33,029 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,030 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,030 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:33,030 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:33,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:33,031 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,034 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,034 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,034 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,034 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,034 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,034 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,034 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:33,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:33,037 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:33,037 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:33,037 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:33,037 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,037 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:33,038 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,038 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,038 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,038 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,038 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,038 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,039 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,039 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,040 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,041 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,042 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,042 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,044 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,044 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,045 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:33,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,044 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,045 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,045 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,045 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:33,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,045 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,046 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,145 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:33,145 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,146 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:33,158 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,158 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:33,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:33,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,160 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:33,160 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:33,161 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,161 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,166 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,166 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,166 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,166 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,166 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,166 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:33,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:33,168 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:33,168 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,168 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:33,168 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:54:33,169 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,170 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,170 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,170 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,171 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,171 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,177 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce 2023-02-09 16:54:33,188 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/f9af0d2a11574aed9cf49a5e78650e29 2023-02-09 16:54:33,192 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce 2023-02-09 16:54:33,195 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:33,196 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/f9af0d2a11574aed9cf49a5e78650e29 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29 2023-02-09 16:54:33,199 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:54:33,200 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 32ms, sequenceid=71, compaction requested=true 2023-02-09 16:54:33,200 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:33,200 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,200 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:33,200 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:33,200 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,200 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,200 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,203 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,203 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,203 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,203 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:33,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fd8d1d6[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:33,205 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:33,205 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,206 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,206 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,206 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:33,207 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,207 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,207 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:33,206 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:33,206 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,207 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,207 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:33,208 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,208 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,208 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,208 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,209 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,209 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,209 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,209 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,209 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,210 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:33,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:33,212 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:33,212 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:33,212 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:33,212 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,212 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,215 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,216 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,216 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,218 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,218 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,218 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,221 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,220 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,221 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,221 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:33,221 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:33,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,221 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,221 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,321 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:33,322 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,322 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:33,333 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,333 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,334 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:33,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:33,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,336 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,336 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,336 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:33,336 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:33,336 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,336 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,337 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,337 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,337 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,337 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,338 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,338 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:33,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,339 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,340 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,340 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,340 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:33,340 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-09 16:54:33,341 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,343 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,343 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,343 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,343 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,343 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,349 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:33,353 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:33,362 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:33,366 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:33,367 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/562b5087b83b46bc961e6ad3e5fa9295 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:33,370 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:33,370 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 16:54:33,371 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/2ca65cc48441403eb4531b526c6fb81b as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:33,374 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:33,374 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 16:54:33,375 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 35ms, sequenceid=83, compaction requested=true 2023-02-09 16:54:33,375 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:33,375 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,375 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:33,375 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:33,375 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,375 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,375 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,377 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,377 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,377 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,377 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:33,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1414b532[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:33,379 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:33,379 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,380 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:33,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,380 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,380 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,381 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,381 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:33,381 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,381 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,380 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,381 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:33,381 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:33,382 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,382 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,382 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,382 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,388 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,388 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,388 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,388 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,388 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,388 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:33,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:33,391 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:33,391 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:33,391 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:33,391 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,391 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,392 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,393 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,393 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,393 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,395 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,395 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,395 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:33,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:33,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,499 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:33,499 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,499 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:33,507 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,507 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:33,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:33,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,509 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,509 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,509 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:33,509 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:33,510 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,510 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,511 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,511 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,511 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,511 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,512 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,512 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,513 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,513 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:33,514 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,514 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,514 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:33,514 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 16:54:33,515 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,516 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,517 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,523 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:33,526 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:33,535 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:33,538 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:33,539 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/3af4582a29a240fd83bdb0312585720f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:33,543 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:33,543 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 16:54:33,543 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/60641890309a4873b3c9855bc9018bbf as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:33,547 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:33,547 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 16:54:33,547 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 33ms, sequenceid=91, compaction requested=true 2023-02-09 16:54:33,548 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:33,548 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,548 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:33,548 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:33,548 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,548 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,548 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,549 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,549 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,549 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,549 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:33,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bb9cf04[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:33,551 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:33,551 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,553 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,553 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,553 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,554 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,554 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:33,553 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:33,555 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,555 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,555 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,555 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,556 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,556 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,556 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,556 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,556 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,556 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:33,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:33,558 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:33,558 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:33,558 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:33,558 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,558 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,560 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,561 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,561 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,561 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,563 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,563 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,563 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,570 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,570 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,571 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,671 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:33,671 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,672 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:33,679 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,679 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,680 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:54:33,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:54:33,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,681 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,682 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,682 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 16:54:33,682 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 16:54:33,682 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,682 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,683 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,683 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,683 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,683 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,684 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,684 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,685 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,685 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:54:33,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:54:33,686 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,686 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. started... 2023-02-09 16:54:33,686 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 16:54:33,687 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,689 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,689 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,689 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,689 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,689 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,696 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:33,700 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:33,708 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:33,712 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:33,712 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/a68995f9b6f44187b6856b9ebd5df5cf as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:33,716 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:33,716 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 16:54:33,717 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/43f700b35ccb43edade157cfb87e34a9 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:33,721 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:33,721 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 16:54:33,721 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 35ms, sequenceid=99, compaction requested=true 2023-02-09 16:54:33,721 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e8ba877d307230da0baa92e22cac26a: 2023-02-09 16:54:33,721 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:33,722 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:54:33,722 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:54:33,722 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,722 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 16:54:33,722 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 16:54:33,723 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,723 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,723 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,723 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 16:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a95e741[Count = 0] remaining members to acquire global barrier 2023-02-09 16:54:33,725 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 16:54:33,725 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,727 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,727 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:54:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:54:33,728 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,728 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,728 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,730 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,730 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 16:54:33,730 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,730 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,730 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,730 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,730 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 16:54:33,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:54:33,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:54:33,733 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 16:54:33,733 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:54:33,733 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:54:33,733 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,733 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:54:33,734 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,734 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,734 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,735 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,735 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,735 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,735 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:54:33,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,737 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,737 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:54:33,737 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 16:54:33,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,740 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:54:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:33,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:54:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:54:33,740 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,741 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,741 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:33,741 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,741 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:33,840 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:54:33,841 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:33,841 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:33,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:54:33,844 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:33,846 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:33,846 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:33,846 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:33,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, UNASSIGN}] 2023-02-09 16:54:33,849 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, UNASSIGN 2023-02-09 16:54:33,849 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=7e8ba877d307230da0baa92e22cac26a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:33,849 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675961673849"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961673849"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961673849"}]},"ts":"1675961673849"} 2023-02-09 16:54:33,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 7e8ba877d307230da0baa92e22cac26a, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:34,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7e8ba877d307230da0baa92e22cac26a, disabling compactions & flushes 2023-02-09 16:54:34,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:34,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:34,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. after waiting 0 ms 2023-02-09 16:54:34,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:34,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7e8ba877d307230da0baa92e22cac26a 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-09 16:54:34,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:34,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:34,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:34,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:34,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf1/ec13414d7c8e4bfdb0c329cb389009d2 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:34,033 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:34,033 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 16:54:34,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.tmp/cf2/8cbbec7fa85c4a329afea87d9bae6d9f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:34,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:34,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 16:54:34,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 36ms, sequenceid=103, compaction requested=true 2023-02-09 16:54:34,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-09 16:54:34,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:34,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. 2023-02-09 16:54:34,043 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7e8ba877d307230da0baa92e22cac26a: Waiting for close lock at 1675961674002Running coprocessor pre-close hooks at 1675961674002Disabling compacts and flushes for region at 1675961674002Disabling writes for close at 1675961674002Obtaining lock to block concurrent updates at 1675961674002Preparing flush snapshotting stores in 7e8ba877d307230da0baa92e22cac26a at 1675961674003 (+1 ms)Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675961674003Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a. at 1675961674003Flushing 7e8ba877d307230da0baa92e22cac26a/cf1: creating writer at 1675961674004 (+1 ms)Flushing 7e8ba877d307230da0baa92e22cac26a/cf1: appending metadata at 1675961674006 (+2 ms)Flushing 7e8ba877d307230da0baa92e22cac26a/cf1: closing flushed file at 1675961674006Flushing 7e8ba877d307230da0baa92e22cac26a/cf2: creating writer at 1675961674018 (+12 ms)Flushing 7e8ba877d307230da0baa92e22cac26a/cf2: appending metadata at 1675961674019 (+1 ms)Flushing 7e8ba877d307230da0baa92e22cac26a/cf2: closing flushed file at 1675961674019Flushing 7e8ba877d307230da0baa92e22cac26a/cf1: reopening flushed file at 1675961674030 (+11 ms)Flushing 7e8ba877d307230da0baa92e22cac26a/cf2: reopening flushed file at 1675961674034 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 7e8ba877d307230da0baa92e22cac26a in 36ms, sequenceid=103, compaction requested=true at 1675961674038 (+4 ms)Writing region close event to WAL at 1675961674040 (+2 ms)Running coprocessor post-close hooks at 1675961674043 (+3 ms)Closed at 1675961674043 2023-02-09 16:54:34,044 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=7e8ba877d307230da0baa92e22cac26a, regionState=CLOSED 2023-02-09 16:54:34,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675961674045"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961674045"}]},"ts":"1675961674045"} 2023-02-09 16:54:34,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-09 16:54:34,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 7e8ba877d307230da0baa92e22cac26a, server=jenkins-hbase4.apache.org,33383,1675961647775 in 197 msec 2023-02-09 16:54:34,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 16:54:34,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7e8ba877d307230da0baa92e22cac26a, UNASSIGN in 200 msec 2023-02-09 16:54:34,064 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=7e8ba877d307230da0baa92e22cac26a, threads=16 2023-02-09 16:54:34,064 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,064 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,064 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,064 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,064 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,065 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,065 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,065 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,065 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,068 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,070 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,070 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,070 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,071 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,076 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,076 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,080 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:34,081 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:34,081 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7, top=false 2023-02-09 16:54:34,086 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83, top=false 2023-02-09 16:54:34,086 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29, top=false 2023-02-09 16:54:34,086 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e, top=false 2023-02-09 16:54:34,088 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:34,092 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:34,093 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:34,093 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:34,093 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce, top=false 2023-02-09 16:54:34,094 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:34,106 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:34,106 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf, top=true 2023-02-09 16:54:34,137 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:34,138 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f, top=true 2023-02-09 16:54:34,138 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:34,138 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f, top=true 2023-02-09 16:54:34,141 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:34,152 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29 for child: 4fa71edecc82d2950cbaa2c51d57f16c, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,156 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e for child: 4fa71edecc82d2950cbaa2c51d57f16c, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,158 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7 for child: 4fa71edecc82d2950cbaa2c51d57f16c, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,160 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:34,160 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf, top=true 2023-02-09 16:54:34,161 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,161 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,161 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:34,161 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9, top=true 2023-02-09 16:54:34,162 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,162 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,168 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248, top=false 2023-02-09 16:54:34,172 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf for child: c9f56ee2a4d5fc619dd62cf9306271f6, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,172 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,172 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,174 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f for child: c9f56ee2a4d5fc619dd62cf9306271f6, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,174 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,174 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,175 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:34,186 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f for child: c9f56ee2a4d5fc619dd62cf9306271f6, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,186 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,187 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83 for child: 4fa71edecc82d2950cbaa2c51d57f16c, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,187 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce for child: 4fa71edecc82d2950cbaa2c51d57f16c, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,204 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf for child: c9f56ee2a4d5fc619dd62cf9306271f6, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,204 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,207 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9 for child: c9f56ee2a4d5fc619dd62cf9306271f6, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,207 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,208 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248 for child: 4fa71edecc82d2950cbaa2c51d57f16c, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,212 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,213 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,213 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:34,213 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2, top=true 2023-02-09 16:54:34,213 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,214 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,214 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:34,216 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,224 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,256 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/.splits/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2 for child: c9f56ee2a4d5fc619dd62cf9306271f6, parent: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,256 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,256 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,260 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,268 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,268 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,271 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,621 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08 for region: 7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,621 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 7e8ba877d307230da0baa92e22cac26a Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:54:34,656 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 16:54:34,658 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 16:54:34,659 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675961674659"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675961674659"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675961674659"}]},"ts":"1675961674659"} 2023-02-09 16:54:34,659 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961674659"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961674659"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961674659"}]},"ts":"1675961674659"} 2023-02-09 16:54:34,659 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961674659"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961674659"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961674659"}]},"ts":"1675961674659"} 2023-02-09 16:54:34,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, ASSIGN}] 2023-02-09 16:54:34,663 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, ASSIGN 2023-02-09 16:54:34,664 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:34,664 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, ASSIGN 2023-02-09 16:54:34,665 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:54:34,814 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:54:34,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=c9f56ee2a4d5fc619dd62cf9306271f6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:34,816 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961674815"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961674815"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961674815"}]},"ts":"1675961674815"} 2023-02-09 16:54:34,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure c9f56ee2a4d5fc619dd62cf9306271f6, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:34,819 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=4fa71edecc82d2950cbaa2c51d57f16c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:34,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961674819"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961674819"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961674819"}]},"ts":"1675961674819"} 2023-02-09 16:54:34,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 4fa71edecc82d2950cbaa2c51d57f16c, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:34,891 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:54:34,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:34,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4fa71edecc82d2950cbaa2c51d57f16c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.', STARTKEY => '', ENDKEY => '60'} 2023-02-09 16:54:34,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:34,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:34,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:34,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:34,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:34,973 INFO [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:34,974 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1 2023-02-09 16:54:34,974 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1 2023-02-09 16:54:34,974 INFO [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa71edecc82d2950cbaa2c51d57f16c columnFamilyName cf1 2023-02-09 16:54:34,983 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:34,984 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295-bottom 2023-02-09 16:54:34,988 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b-bottom 2023-02-09 16:54:34,993 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1-bottom 2023-02-09 16:54:34,997 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68-bottom 2023-02-09 16:54:34,999 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce 2023-02-09 16:54:35,003 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e 2023-02-09 16:54:35,005 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248 2023-02-09 16:54:35,006 INFO [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(310): Store=4fa71edecc82d2950cbaa2c51d57f16c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:35,006 INFO [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,006 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2 2023-02-09 16:54:35,007 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2 2023-02-09 16:54:35,007 INFO [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa71edecc82d2950cbaa2c51d57f16c columnFamilyName cf2 2023-02-09 16:54:35,018 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0-bottom 2023-02-09 16:54:35,022 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300-bottom 2023-02-09 16:54:35,024 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,025 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b-bottom 2023-02-09 16:54:35,029 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08-bottom 2023-02-09 16:54:35,032 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7 2023-02-09 16:54:35,034 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83 2023-02-09 16:54:35,037 DEBUG [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29 2023-02-09 16:54:35,037 INFO [StoreOpener-4fa71edecc82d2950cbaa2c51d57f16c-1] regionserver.HStore(310): Store=4fa71edecc82d2950cbaa2c51d57f16c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:35,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:35,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4fa71edecc82d2950cbaa2c51d57f16c; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69000305, jitterRate=0.028184667229652405}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:35,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4fa71edecc82d2950cbaa2c51d57f16c: Running coprocessor pre-open hook at 1675961674972Writing region info on filesystem at 1675961674972Initializing all the Stores at 1675961674973 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961674973Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961674973Cleaning up temporary data from old regions at 1675961675039 (+66 ms)Cleaning up detritus from prior splits at 1675961675040 (+1 ms)Running coprocessor post-open hooks at 1675961675042 (+2 ms)Region opened successfully at 1675961675042 2023-02-09 16:54:35,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c., pid=78, masterSystemTime=1675961674968 2023-02-09 16:54:35,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:35,044 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:35,044 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:35,044 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,044 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1912): 4fa71edecc82d2950cbaa2c51d57f16c/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:35,044 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,045 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1912): 4fa71edecc82d2950cbaa2c51d57f16c/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:35,045 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4fa71edecc82d2950cbaa2c51d57f16c/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,045 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4fa71edecc82d2950cbaa2c51d57f16c/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,045 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp, totalSize=35.0 K 2023-02-09 16:54:35,045 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp, totalSize=35.0 K 2023-02-09 16:54:35,045 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961672226 2023-02-09 16:54:35,046 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961672226 2023-02-09 16:54:35,046 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961672409 2023-02-09 16:54:35,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,046 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961672409 2023-02-09 16:54:35,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=4fa71edecc82d2950cbaa2c51d57f16c, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:35,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c9f56ee2a4d5fc619dd62cf9306271f6, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.', STARTKEY => '60', ENDKEY => ''} 2023-02-09 16:54:35,046 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961675046"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961675046"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961675046"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961675046"}]},"ts":"1675961675046"} 2023-02-09 16:54:35,046 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961672595 2023-02-09 16:54:35,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:35,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:35,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,048 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961672595 2023-02-09 16:54:35,048 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961672778 2023-02-09 16:54:35,048 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961672778 2023-02-09 16:54:35,048 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961672968 2023-02-09 16:54:35,049 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961672968 2023-02-09 16:54:35,049 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961673147 2023-02-09 16:54:35,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-09 16:54:35,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 4fa71edecc82d2950cbaa2c51d57f16c, server=jenkins-hbase4.apache.org,33383,1675961647775 in 228 msec 2023-02-09 16:54:35,049 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961673147 2023-02-09 16:54:35,049 INFO [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,049 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961673323 2023-02-09 16:54:35,050 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961673323 2023-02-09 16:54:35,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, ASSIGN in 387 msec 2023-02-09 16:54:35,052 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1 2023-02-09 16:54:35,052 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1 2023-02-09 16:54:35,053 INFO [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9f56ee2a4d5fc619dd62cf9306271f6 columnFamilyName cf1 2023-02-09 16:54:35,069 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] throttle.PressureAwareThroughputController(145): 4fa71edecc82d2950cbaa2c51d57f16c#cf2#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,073 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,073 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295-top 2023-02-09 16:54:35,074 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4fa71edecc82d2950cbaa2c51d57f16c#cf1#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,078 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b-top 2023-02-09 16:54:35,084 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1-top 2023-02-09 16:54:35,088 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68-top 2023-02-09 16:54:35,088 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp/cf2/8dadabf4468d4b2ca60956a2b744e5e4 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/8dadabf4468d4b2ca60956a2b744e5e4 2023-02-09 16:54:35,089 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp/cf1/bab0707209ed4e50aa5c5dd5f729134f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/bab0707209ed4e50aa5c5dd5f729134f 2023-02-09 16:54:35,092 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:35,092 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:35,094 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4fa71edecc82d2950cbaa2c51d57f16c/cf2 of 4fa71edecc82d2950cbaa2c51d57f16c into 8dadabf4468d4b2ca60956a2b744e5e4(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,094 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4fa71edecc82d2950cbaa2c51d57f16c: 2023-02-09 16:54:35,094 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c., storeName=4fa71edecc82d2950cbaa2c51d57f16c/cf2, priority=9, startTime=1675961675043; duration=0sec 2023-02-09 16:54:35,094 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,095 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4fa71edecc82d2950cbaa2c51d57f16c/cf1 of 4fa71edecc82d2950cbaa2c51d57f16c into bab0707209ed4e50aa5c5dd5f729134f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,096 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4fa71edecc82d2950cbaa2c51d57f16c: 2023-02-09 16:54:35,096 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c., storeName=4fa71edecc82d2950cbaa2c51d57f16c/cf1, priority=9, startTime=1675961675043; duration=0sec 2023-02-09 16:54:35,096 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,097 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:35,097 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:35,101 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:35,101 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:35,101 INFO [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(310): Store=c9f56ee2a4d5fc619dd62cf9306271f6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:35,101 INFO [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,102 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2 2023-02-09 16:54:35,102 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2 2023-02-09 16:54:35,103 INFO [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9f56ee2a4d5fc619dd62cf9306271f6 columnFamilyName cf2 2023-02-09 16:54:35,112 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0-top 2023-02-09 16:54:35,114 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300-top 2023-02-09 16:54:35,117 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,117 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b-top 2023-02-09 16:54:35,120 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08-top 2023-02-09 16:54:35,122 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:35,123 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:35,125 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:35,125 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:35,128 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:35,128 DEBUG [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:35,129 INFO [StoreOpener-c9f56ee2a4d5fc619dd62cf9306271f6-1] regionserver.HStore(310): Store=c9f56ee2a4d5fc619dd62cf9306271f6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:35,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:54:35,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c9f56ee2a4d5fc619dd62cf9306271f6; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73358771, jitterRate=0.09313087165355682}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:54:35,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c9f56ee2a4d5fc619dd62cf9306271f6: Running coprocessor pre-open hook at 1675961675047Writing region info on filesystem at 1675961675047Initializing all the Stores at 1675961675049 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961675049Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961675049Cleaning up temporary data from old regions at 1675961675130 (+81 ms)Cleaning up detritus from prior splits at 1675961675131 (+1 ms)Running coprocessor post-open hooks at 1675961675133 (+2 ms)Region opened successfully at 1675961675133 2023-02-09 16:54:35,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6., pid=77, masterSystemTime=1675961674968 2023-02-09 16:54:35,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:35,135 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:35,135 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:54:35,135 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,135 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1912): c9f56ee2a4d5fc619dd62cf9306271f6/cf1 is initiating minor compaction (all files) 2023-02-09 16:54:35,135 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,135 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1912): c9f56ee2a4d5fc619dd62cf9306271f6/cf2 is initiating minor compaction (all files) 2023-02-09 16:54:35,135 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c9f56ee2a4d5fc619dd62cf9306271f6/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,135 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c9f56ee2a4d5fc619dd62cf9306271f6/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,135 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp, totalSize=34.8 K 2023-02-09 16:54:35,135 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp, totalSize=34.8 K 2023-02-09 16:54:35,136 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961672226 2023-02-09 16:54:35,136 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961672226 2023-02-09 16:54:35,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,136 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961672409 2023-02-09 16:54:35,136 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961672409 2023-02-09 16:54:35,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,137 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961672595 2023-02-09 16:54:35,137 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961672595 2023-02-09 16:54:35,137 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=c9f56ee2a4d5fc619dd62cf9306271f6, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:35,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961675137"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961675137"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961675137"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961675137"}]},"ts":"1675961675137"} 2023-02-09 16:54:35,137 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675961673323 2023-02-09 16:54:35,137 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675961673323 2023-02-09 16:54:35,137 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,138 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,138 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,138 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,138 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,139 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-09 16:54:35,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure c9f56ee2a4d5fc619dd62cf9306271f6, server=jenkins-hbase4.apache.org,33383,1675961647775 in 321 msec 2023-02-09 16:54:35,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-09 16:54:35,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, ASSIGN in 479 msec 2023-02-09 16:54:35,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 in 1.2990 sec 2023-02-09 16:54:35,153 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] throttle.PressureAwareThroughputController(145): c9f56ee2a4d5fc619dd62cf9306271f6#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,153 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] throttle.PressureAwareThroughputController(145): c9f56ee2a4d5fc619dd62cf9306271f6#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,170 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 304bc9e9a3d541f0ae5946a272414228: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,170 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 474a36ab5ae343eabe91e15c2a68d9e1: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,170 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp/cf2/304bc9e9a3d541f0ae5946a272414228 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/304bc9e9a3d541f0ae5946a272414228 2023-02-09 16:54:35,171 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp/cf1/474a36ab5ae343eabe91e15c2a68d9e1 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/474a36ab5ae343eabe91e15c2a68d9e1 2023-02-09 16:54:35,174 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 304bc9e9a3d541f0ae5946a272414228: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,174 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 474a36ab5ae343eabe91e15c2a68d9e1: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,175 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c9f56ee2a4d5fc619dd62cf9306271f6/cf2 of c9f56ee2a4d5fc619dd62cf9306271f6 into 304bc9e9a3d541f0ae5946a272414228(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,175 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c9f56ee2a4d5fc619dd62cf9306271f6/cf1 of c9f56ee2a4d5fc619dd62cf9306271f6 into 474a36ab5ae343eabe91e15c2a68d9e1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,175 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c9f56ee2a4d5fc619dd62cf9306271f6: 2023-02-09 16:54:35,175 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c9f56ee2a4d5fc619dd62cf9306271f6: 2023-02-09 16:54:35,175 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6., storeName=c9f56ee2a4d5fc619dd62cf9306271f6/cf2, priority=9, startTime=1675961675135; duration=0sec 2023-02-09 16:54:35,175 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6., storeName=c9f56ee2a4d5fc619dd62cf9306271f6/cf1, priority=9, startTime=1675961675133; duration=0sec 2023-02-09 16:54:35,175 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,176 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,355 DEBUG [Listener at localhost/39015] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 16:54:35,358 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54048, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 16:54:35,358 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,359 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 16:54:35,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HStore(1912): 4fa71edecc82d2950cbaa2c51d57f16c/cf1 is initiating major compaction (all files) 2023-02-09 16:54:35,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1d14f241; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,360 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4fa71edecc82d2950cbaa2c51d57f16c/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 16:54:35,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HStore(1912): 4fa71edecc82d2950cbaa2c51d57f16c/cf2 is initiating major compaction (all files) 2023-02-09 16:54:35,360 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/bab0707209ed4e50aa5c5dd5f729134f] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp, totalSize=6.8 K 2023-02-09 16:54:35,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@43c383a2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,360 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4fa71edecc82d2950cbaa2c51d57f16c/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,360 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/8dadabf4468d4b2ca60956a2b744e5e4] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp, totalSize=6.8 K 2023-02-09 16:54:35,360 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting bab0707209ed4e50aa5c5dd5f729134f, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961672226 2023-02-09 16:54:35,360 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 8dadabf4468d4b2ca60956a2b744e5e4, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961672226 2023-02-09 16:54:35,361 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 16:54:35,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.HStore(1912): c9f56ee2a4d5fc619dd62cf9306271f6/cf1 is initiating major compaction (all files) 2023-02-09 16:54:35,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1ff74e6c; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:35,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 16:54:35,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.HStore(1912): c9f56ee2a4d5fc619dd62cf9306271f6/cf2 is initiating major compaction (all files) 2023-02-09 16:54:35,362 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1f23b6c4; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 16:54:35,362 INFO [Listener at localhost/39015] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 16:54:35,365 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] throttle.PressureAwareThroughputController(145): 4fa71edecc82d2950cbaa2c51d57f16c#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,365 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4fa71edecc82d2950cbaa2c51d57f16c#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,372 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. is not mergeable because it has references 2023-02-09 16:54:35,372 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. is not mergeable because it has references 2023-02-09 16:54:35,381 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp/cf1/261c0e32c2f94487ac40e0bbcc537269 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/261c0e32c2f94487ac40e0bbcc537269 2023-02-09 16:54:35,381 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/.tmp/cf2/ee39cb7183e8420490974fc2da731444 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/ee39cb7183e8420490974fc2da731444 2023-02-09 16:54:35,385 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 4fa71edecc82d2950cbaa2c51d57f16c/cf1 of 4fa71edecc82d2950cbaa2c51d57f16c into 261c0e32c2f94487ac40e0bbcc537269(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,385 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4fa71edecc82d2950cbaa2c51d57f16c: 2023-02-09 16:54:35,386 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 4fa71edecc82d2950cbaa2c51d57f16c/cf2 of 4fa71edecc82d2950cbaa2c51d57f16c into ee39cb7183e8420490974fc2da731444(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,386 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4fa71edecc82d2950cbaa2c51d57f16c: 2023-02-09 16:54:35,387 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675961675359; duration=0sec 2023-02-09 16:54:35,387 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675961675360; duration=0sec 2023-02-09 16:54:35,387 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 16:54:35,387 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:54:35,387 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c9f56ee2a4d5fc619dd62cf9306271f6/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,387 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c9f56ee2a4d5fc619dd62cf9306271f6/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,387 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/304bc9e9a3d541f0ae5946a272414228] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp, totalSize=5.0 K 2023-02-09 16:54:35,387 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/474a36ab5ae343eabe91e15c2a68d9e1] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp, totalSize=5.0 K 2023-02-09 16:54:35,387 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 304bc9e9a3d541f0ae5946a272414228, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,388 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 474a36ab5ae343eabe91e15c2a68d9e1, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 16:54:35,390 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] throttle.PressureAwareThroughputController(145): c9f56ee2a4d5fc619dd62cf9306271f6#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,390 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] throttle.PressureAwareThroughputController(145): c9f56ee2a4d5fc619dd62cf9306271f6#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:54:35,404 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for bdcb2e86eb104d868a37ea465e01bf6b: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,404 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d3a21a88e55a432c9e0e9a65351f9fd0: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,404 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp/cf2/bdcb2e86eb104d868a37ea465e01bf6b as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/bdcb2e86eb104d868a37ea465e01bf6b 2023-02-09 16:54:35,405 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/.tmp/cf1/d3a21a88e55a432c9e0e9a65351f9fd0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/d3a21a88e55a432c9e0e9a65351f9fd0 2023-02-09 16:54:35,408 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for bdcb2e86eb104d868a37ea465e01bf6b: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,409 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for d3a21a88e55a432c9e0e9a65351f9fd0: NONE, but ROW specified in column family configuration 2023-02-09 16:54:35,409 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c9f56ee2a4d5fc619dd62cf9306271f6/cf2 of c9f56ee2a4d5fc619dd62cf9306271f6 into bdcb2e86eb104d868a37ea465e01bf6b(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,409 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c9f56ee2a4d5fc619dd62cf9306271f6: 2023-02-09 16:54:35,409 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675961675362; duration=0sec 2023-02-09 16:54:35,409 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,409 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c9f56ee2a4d5fc619dd62cf9306271f6/cf1 of c9f56ee2a4d5fc619dd62cf9306271f6 into d3a21a88e55a432c9e0e9a65351f9fd0(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:54:35,409 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c9f56ee2a4d5fc619dd62cf9306271f6: 2023-02-09 16:54:35,410 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675961675362; duration=0sec 2023-02-09 16:54:35,410 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:54:35,481 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. is not mergeable because it has references 2023-02-09 16:54:35,481 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33383] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. is not mergeable because it has references 2023-02-09 16:54:35,487 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-09 16:54:35,489 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:35,489 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,489 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,490 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,490 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,491 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,491 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,493 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,493 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 16:54:35,494 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961675494"}]},"ts":"1675961675494"} 2023-02-09 16:54:35,496 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-09 16:54:35,497 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-09 16:54:35,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, UNASSIGN}] 2023-02-09 16:54:35,499 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, UNASSIGN 2023-02-09 16:54:35,499 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=c9f56ee2a4d5fc619dd62cf9306271f6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:35,499 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961675499"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961675499"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961675499"}]},"ts":"1675961675499"} 2023-02-09 16:54:35,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure c9f56ee2a4d5fc619dd62cf9306271f6, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:35,502 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, UNASSIGN 2023-02-09 16:54:35,503 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=4fa71edecc82d2950cbaa2c51d57f16c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:54:35,503 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961675503"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961675503"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961675503"}]},"ts":"1675961675503"} 2023-02-09 16:54:35,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 4fa71edecc82d2950cbaa2c51d57f16c, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:54:35,594 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,594 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 16:54:35,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4fa71edecc82d2950cbaa2c51d57f16c, disabling compactions & flushes 2023-02-09 16:54:35,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. after waiting 0 ms 2023-02-09 16:54:35,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,653 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/bab0707209ed4e50aa5c5dd5f729134f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295-bottom] to archive 2023-02-09 16:54:35,654 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:35,655 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,656 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,657 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,658 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-9afad62c8d9f473bbe0180074ce2336e 2023-02-09 16:54:35,659 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-c15e2085e3114933b9082753f22bc248 2023-02-09 16:54:35,660 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-32e51bcb8eb440f5b8abfb0d6adde5ce 2023-02-09 16:54:35,661 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/bab0707209ed4e50aa5c5dd5f729134f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/bab0707209ed4e50aa5c5dd5f729134f 2023-02-09 16:54:35,662 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,664 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/8dadabf4468d4b2ca60956a2b744e5e4, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b-bottom] to archive 2023-02-09 16:54:35,664 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:35,665 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,666 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,667 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,668 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-4f22fb1a9f194088ad314974946111b7 2023-02-09 16:54:35,669 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-7f8e469ea8d142f7bf90a331d8c5ad83 2023-02-09 16:54:35,670 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-f9af0d2a11574aed9cf49a5e78650e29 2023-02-09 16:54:35,671 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/8dadabf4468d4b2ca60956a2b744e5e4 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/8dadabf4468d4b2ca60956a2b744e5e4 2023-02-09 16:54:35,672 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 16:54:35,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:35,676 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c. 2023-02-09 16:54:35,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4fa71edecc82d2950cbaa2c51d57f16c: Waiting for close lock at 1675961675652Running coprocessor pre-close hooks at 1675961675652Disabling compacts and flushes for region at 1675961675652Disabling writes for close at 1675961675652Writing region close event to WAL at 1675961675673 (+21 ms)Running coprocessor post-close hooks at 1675961675676 (+3 ms)Closed at 1675961675676 2023-02-09 16:54:35,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c9f56ee2a4d5fc619dd62cf9306271f6, disabling compactions & flushes 2023-02-09 16:54:35,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. after waiting 0 ms 2023-02-09 16:54:35,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,678 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=4fa71edecc82d2950cbaa2c51d57f16c, regionState=CLOSED 2023-02-09 16:54:35,678 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961675678"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961675678"}]},"ts":"1675961675678"} 2023-02-09 16:54:35,679 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/474a36ab5ae343eabe91e15c2a68d9e1, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2] to archive 2023-02-09 16:54:35,680 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:35,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 16:54:35,681 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/ffd216f8f03c43acaddfb74ba2128f68.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 4fa71edecc82d2950cbaa2c51d57f16c, server=jenkins-hbase4.apache.org,33383,1675961647775 in 176 msec 2023-02-09 16:54:35,684 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/b5adafa95567426fb5667dd81cd2fbf1.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,685 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/84423ee5535c4a86b9e26dc58df7c83b.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,687 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/562b5087b83b46bc961e6ad3e5fa9295.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=4fa71edecc82d2950cbaa2c51d57f16c, UNASSIGN in 184 msec 2023-02-09 16:54:35,688 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:35,689 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:35,690 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/474a36ab5ae343eabe91e15c2a68d9e1 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/474a36ab5ae343eabe91e15c2a68d9e1 2023-02-09 16:54:35,692 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:35,694 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/304bc9e9a3d541f0ae5946a272414228, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f] to archive 2023-02-09 16:54:35,694 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:54:35,696 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/09437cf1d1a3492eac56f901331af0e0.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,697 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/3ba98099972245b69922170361b38d08.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,698 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/11b980d0a89d4af6a21d9d02cec64300.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,700 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/2ca65cc48441403eb4531b526c6fb81b.7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,701 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:35,702 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:35,703 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/304bc9e9a3d541f0ae5946a272414228 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/304bc9e9a3d541f0ae5946a272414228 2023-02-09 16:54:35,705 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/testSplitTableRegionDeletedRowsDaughter=7e8ba877d307230da0baa92e22cac26a-8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:35,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 16:54:35,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:35,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6. 2023-02-09 16:54:35,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c9f56ee2a4d5fc619dd62cf9306271f6: Waiting for close lock at 1675961675678Running coprocessor pre-close hooks at 1675961675678Disabling compacts and flushes for region at 1675961675678Disabling writes for close at 1675961675678Writing region close event to WAL at 1675961675714 (+36 ms)Running coprocessor post-close hooks at 1675961675717 (+3 ms)Closed at 1675961675717 2023-02-09 16:54:35,718 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,719 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=c9f56ee2a4d5fc619dd62cf9306271f6, regionState=CLOSED 2023-02-09 16:54:35,719 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675961675719"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961675719"}]},"ts":"1675961675719"} 2023-02-09 16:54:35,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-09 16:54:35,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure c9f56ee2a4d5fc619dd62cf9306271f6, server=jenkins-hbase4.apache.org,33383,1675961647775 in 220 msec 2023-02-09 16:54:35,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-09 16:54:35,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c9f56ee2a4d5fc619dd62cf9306271f6, UNASSIGN in 223 msec 2023-02-09 16:54:35,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961675723"}]},"ts":"1675961675723"} 2023-02-09 16:54:35,724 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-09 16:54:35,726 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-09 16:54:35,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 235 msec 2023-02-09 16:54:35,795 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,796 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 16:54:35,797 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-09 16:54:35,797 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,797 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,797 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,800 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,800 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,800 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,800 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 16:54:35,805 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,805 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,805 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,807 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/recovered.edits] 2023-02-09 16:54:35,807 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/recovered.edits] 2023-02-09 16:54:35,807 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/recovered.edits] 2023-02-09 16:54:35,810 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/d3a21a88e55a432c9e0e9a65351f9fd0 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf1/d3a21a88e55a432c9e0e9a65351f9fd0 2023-02-09 16:54:35,810 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/261c0e32c2f94487ac40e0bbcc537269 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf1/261c0e32c2f94487ac40e0bbcc537269 2023-02-09 16:54:35,810 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/32e51bcb8eb440f5b8abfb0d6adde5ce 2023-02-09 16:54:35,812 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/3af4582a29a240fd83bdb0312585720f 2023-02-09 16:54:35,813 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/bdcb2e86eb104d868a37ea465e01bf6b to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/cf2/bdcb2e86eb104d868a37ea465e01bf6b 2023-02-09 16:54:35,813 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/ee39cb7183e8420490974fc2da731444 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/cf2/ee39cb7183e8420490974fc2da731444 2023-02-09 16:54:35,813 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/562b5087b83b46bc961e6ad3e5fa9295 2023-02-09 16:54:35,815 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/84423ee5535c4a86b9e26dc58df7c83b 2023-02-09 16:54:35,817 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/recovered.edits/113.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c/recovered.edits/113.seqid 2023-02-09 16:54:35,817 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/9afad62c8d9f473bbe0180074ce2336e 2023-02-09 16:54:35,817 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/recovered.edits/113.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6/recovered.edits/113.seqid 2023-02-09 16:54:35,817 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/4fa71edecc82d2950cbaa2c51d57f16c 2023-02-09 16:54:35,818 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:54:35,818 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/a68995f9b6f44187b6856b9ebd5df5cf 2023-02-09 16:54:35,820 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/b5adafa95567426fb5667dd81cd2fbf1 2023-02-09 16:54:35,821 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/c15e2085e3114933b9082753f22bc248 2023-02-09 16:54:35,823 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ec13414d7c8e4bfdb0c329cb389009d2 2023-02-09 16:54:35,824 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf1/ffd216f8f03c43acaddfb74ba2128f68 2023-02-09 16:54:35,826 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/09437cf1d1a3492eac56f901331af0e0 2023-02-09 16:54:35,827 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/11b980d0a89d4af6a21d9d02cec64300 2023-02-09 16:54:35,828 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/2ca65cc48441403eb4531b526c6fb81b 2023-02-09 16:54:35,829 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/3ba98099972245b69922170361b38d08 2023-02-09 16:54:35,831 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/43f700b35ccb43edade157cfb87e34a9 2023-02-09 16:54:35,832 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/4f22fb1a9f194088ad314974946111b7 2023-02-09 16:54:35,833 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/60641890309a4873b3c9855bc9018bbf 2023-02-09 16:54:35,834 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/7f8e469ea8d142f7bf90a331d8c5ad83 2023-02-09 16:54:35,835 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/8cbbec7fa85c4a329afea87d9bae6d9f 2023-02-09 16:54:35,836 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/cf2/f9af0d2a11574aed9cf49a5e78650e29 2023-02-09 16:54:35,839 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/recovered.edits/106.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a/recovered.edits/106.seqid 2023-02-09 16:54:35,839 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7e8ba877d307230da0baa92e22cac26a 2023-02-09 16:54:35,839 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 16:54:35,841 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,843 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-09 16:54:35,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-09 16:54:35,846 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,846 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-09 16:54:35,846 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961675846"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:35,846 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961675846"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:35,846 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961675846"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:35,848 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:54:35,848 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7e8ba877d307230da0baa92e22cac26a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675961671722.7e8ba877d307230da0baa92e22cac26a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4fa71edecc82d2950cbaa2c51d57f16c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675961673844.4fa71edecc82d2950cbaa2c51d57f16c.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => c9f56ee2a4d5fc619dd62cf9306271f6, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675961673844.c9f56ee2a4d5fc619dd62cf9306271f6.', STARTKEY => '60', ENDKEY => ''}] 2023-02-09 16:54:35,848 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-09 16:54:35,848 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961675848"}]},"ts":"9223372036854775807"} 2023-02-09 16:54:35,850 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-09 16:54:35,851 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:54:35,852 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 53 msec 2023-02-09 16:54:35,901 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,901 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 16:54:35,902 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-09 16:54:35,917 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 409) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39428 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39862 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39422 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:32852 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:32788 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:47742 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39928 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39792 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39806 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:47716 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39858 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:47710 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:32828 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:32842 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39866 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:33383-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_831607815_15 at /127.0.0.1:39874 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x82ecaf6-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=860 (was 858) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=167 (was 181), ProcessCount=168 (was 168), AvailableMemoryMB=3113 (was 3184) 2023-02-09 16:54:35,932 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=860, MaxFileDescriptor=60000, SystemLoadAverage=167, ProcessCount=168, AvailableMemoryMB=3112 2023-02-09 16:54:35,932 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:54:35,932 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:35,932 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:35,934 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:54:35,935 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:54:35,936 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:54:35,937 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:54:35,938 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:54:35,939 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:35,940 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 empty. 2023-02-09 16:54:35,940 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:35,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 16:54:35,949 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:35,950 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 82b4da9176b3dc6382be1c3c41771bc2, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:54:35,956 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:35,956 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 82b4da9176b3dc6382be1c3c41771bc2, disabling compactions & flushes 2023-02-09 16:54:35,956 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:35,956 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:35,956 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. after waiting 0 ms 2023-02-09 16:54:35,956 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:35,957 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:35,957 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 82b4da9176b3dc6382be1c3c41771bc2: Waiting for close lock at 1675961675956Disabling compacts and flushes for region at 1675961675956Disabling writes for close at 1675961675956Writing region close event to WAL at 1675961675957 (+1 ms)Closed at 1675961675957 2023-02-09 16:54:35,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:54:35,959 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961675959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961675959"}]},"ts":"1675961675959"} 2023-02-09 16:54:35,960 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:54:35,961 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:54:35,961 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961675961"}]},"ts":"1675961675961"} 2023-02-09 16:54:35,962 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-09 16:54:35,965 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:35,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:35,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:35,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:35,966 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:54:35,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, ASSIGN}] 2023-02-09 16:54:35,967 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, ASSIGN 2023-02-09 16:54:35,968 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37073,1675961647745; forceNewPlan=false, retain=false 2023-02-09 16:54:36,118 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:36,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:36,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961676121"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961676121"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961676121"}]},"ts":"1675961676121"} 2023-02-09 16:54:36,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:36,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 82b4da9176b3dc6382be1c3c41771bc2, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:36,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:36,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:36,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,278 INFO [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,279 DEBUG [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1 2023-02-09 16:54:36,279 DEBUG [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1 2023-02-09 16:54:36,280 INFO [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82b4da9176b3dc6382be1c3c41771bc2 columnFamilyName cf1 2023-02-09 16:54:36,280 INFO [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] regionserver.HStore(310): Store=82b4da9176b3dc6382be1c3c41771bc2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:36,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:54:36,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 82b4da9176b3dc6382be1c3c41771bc2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61119641, jitterRate=-0.08924637734889984}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 16:54:36,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 82b4da9176b3dc6382be1c3c41771bc2: Running coprocessor pre-open hook at 1675961676277Writing region info on filesystem at 1675961676277Initializing all the Stores at 1675961676278 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961676278Cleaning up temporary data from old regions at 1675961676281 (+3 ms)Cleaning up detritus from prior splits at 1675961676282 (+1 ms)Running coprocessor post-open hooks at 1675961676285 (+3 ms)Region opened successfully at 1675961676285 2023-02-09 16:54:36,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2., pid=87, masterSystemTime=1675961676274 2023-02-09 16:54:36,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:36,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961676287"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961676287"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961676287"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961676287"}]},"ts":"1675961676287"} 2023-02-09 16:54:36,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-09 16:54:36,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745 in 167 msec 2023-02-09 16:54:36,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-09 16:54:36,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, ASSIGN in 324 msec 2023-02-09 16:54:36,292 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:54:36,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961676292"}]},"ts":"1675961676292"} 2023-02-09 16:54:36,293 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-09 16:54:36,298 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:54:36,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 363 msec 2023-02-09 16:54:36,437 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:36,437 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:36,439 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:54:36,439 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:54:36,445 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 16:54:36,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:54:36,463 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 16:54:36,464 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 16:54:36,464 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 16:54:36,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-09 16:54:36,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, REOPEN/MOVE}] 2023-02-09 16:54:36,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, REOPEN/MOVE 2023-02-09 16:54:36,479 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:36,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961676479"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961676479"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961676479"}]},"ts":"1675961676479"} 2023-02-09 16:54:36,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:36,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 82b4da9176b3dc6382be1c3c41771bc2, disabling compactions & flushes 2023-02-09 16:54:36,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. after waiting 0 ms 2023-02-09 16:54:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 16:54:36,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:54:36,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 82b4da9176b3dc6382be1c3c41771bc2: Waiting for close lock at 1675961676632Running coprocessor pre-close hooks at 1675961676632Disabling compacts and flushes for region at 1675961676632Disabling writes for close at 1675961676633 (+1 ms)Writing region close event to WAL at 1675961676634 (+1 ms)Running coprocessor post-close hooks at 1675961676637 (+3 ms)Closed at 1675961676637 2023-02-09 16:54:36,637 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 82b4da9176b3dc6382be1c3c41771bc2 to self. 2023-02-09 16:54:36,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,639 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=CLOSED 2023-02-09 16:54:36,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961676638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961676638"}]},"ts":"1675961676638"} 2023-02-09 16:54:36,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-09 16:54:36,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745 in 159 msec 2023-02-09 16:54:36,642 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37073,1675961647745; forceNewPlan=false, retain=true 2023-02-09 16:54:36,792 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:54:36,792 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:36,793 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961676792"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961676792"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961676792"}]},"ts":"1675961676792"} 2023-02-09 16:54:36,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:54:36,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 82b4da9176b3dc6382be1c3c41771bc2, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:54:36,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:36,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:36,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,950 INFO [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,951 DEBUG [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1 2023-02-09 16:54:36,951 DEBUG [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1 2023-02-09 16:54:36,951 INFO [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82b4da9176b3dc6382be1c3c41771bc2 columnFamilyName cf1 2023-02-09 16:54:36,952 INFO [StoreOpener-82b4da9176b3dc6382be1c3c41771bc2-1] regionserver.HStore(310): Store=82b4da9176b3dc6382be1c3c41771bc2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:54:36,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:54:36,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 82b4da9176b3dc6382be1c3c41771bc2; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72694887, jitterRate=0.08323822915554047}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 16:54:36,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 82b4da9176b3dc6382be1c3c41771bc2: Running coprocessor pre-open hook at 1675961676949Writing region info on filesystem at 1675961676949Initializing all the Stores at 1675961676950 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961676950Cleaning up temporary data from old regions at 1675961676954 (+4 ms)Cleaning up detritus from prior splits at 1675961676954Running coprocessor post-open hooks at 1675961676956 (+2 ms)Region opened successfully at 1675961676956 2023-02-09 16:54:36,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2., pid=92, masterSystemTime=1675961676946 2023-02-09 16:54:36,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:54:36,959 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:54:36,959 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961676959"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961676959"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961676959"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961676959"}]},"ts":"1675961676959"} 2023-02-09 16:54:36,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-09 16:54:36,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745 in 166 msec 2023-02-09 16:54:36,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-09 16:54:36,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, REOPEN/MOVE in 483 msec 2023-02-09 16:54:36,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-09 16:54:36,965 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 498 msec 2023-02-09 16:54:36,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, ASSIGN}] 2023-02-09 16:54:36,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, ASSIGN 2023-02-09 16:54:36,970 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 16:54:37,120 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:54:37,121 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:54:37,121 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:54:37,121 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:54:37,121 DEBUG [jenkins-hbase4:35255] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-09 16:54:37,123 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=aa6a554ecfe82c532d26eba502eca4e9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:54:37,123 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961677123"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675961677123"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675961677123"}]},"ts":"1675961677123"} 2023-02-09 16:54:37,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure aa6a554ecfe82c532d26eba502eca4e9, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:54:37,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:54:37,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aa6a554ecfe82c532d26eba502eca4e9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-09 16:54:37,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:54:37,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica aa6a554ecfe82c532d26eba502eca4e9 2023-02-09 16:54:37,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:54:37,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aa6a554ecfe82c532d26eba502eca4e9 2023-02-09 16:54:37,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aa6a554ecfe82c532d26eba502eca4e9 2023-02-09 16:54:37,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:37,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:38,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,095 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-09 16:54:39,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:39,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,474 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:54:40,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:40,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:41,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:42,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:43,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:44,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:45,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:46,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:47,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:48,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:49,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:50,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:51,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:52,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:53,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:54,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:55,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:56,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:57,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:58,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:54:59,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:00,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:01,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:02,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:03,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:04,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:05,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:06,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:07,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:08,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,092 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 16:55:09,095 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 16:55:09,099 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 16:55:09,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:09,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:10,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:11,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:12,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:13,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:14,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:15,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:16,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:17,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:18,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:19,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:20,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:21,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:22,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:23,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:24,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:25,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:26,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:27,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:28,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:29,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:30,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:31,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:32,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:33,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:34,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:35,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,555 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:36,555 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:36,558 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:36,558 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:36,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 16:55:36,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,659 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:36,659 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:36,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 16:55:36,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:36,860 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:36,860 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:36,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 16:55:36,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,161 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:37,161 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:37,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 16:55:37,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,662 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:37,662 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:37,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 16:55:37,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:37,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,664 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:38,665 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:38,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:38,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:38,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:38,668 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 82b4da9176b3dc6382be1c3c41771bc2, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:38,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc exec-time=6 msec 2023-02-09 16:55:38,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:38,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 16:55:39,007 INFO [StoreOpener-aa6a554ecfe82c532d26eba502eca4e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region aa6a554ecfe82c532d26eba502eca4e9 2023-02-09 16:55:39,010 DEBUG [StoreOpener-aa6a554ecfe82c532d26eba502eca4e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1 2023-02-09 16:55:39,010 DEBUG [StoreOpener-aa6a554ecfe82c532d26eba502eca4e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1 2023-02-09 16:55:39,011 INFO [StoreOpener-aa6a554ecfe82c532d26eba502eca4e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa6a554ecfe82c532d26eba502eca4e9 columnFamilyName cf1 2023-02-09 16:55:39,012 INFO [StoreOpener-aa6a554ecfe82c532d26eba502eca4e9-1] regionserver.HStore(310): Store=aa6a554ecfe82c532d26eba502eca4e9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:39,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aa6a554ecfe82c532d26eba502eca4e9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74787158, jitterRate=0.1144154965877533}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 16:55:39,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aa6a554ecfe82c532d26eba502eca4e9: Running coprocessor pre-open hook at 1675961677285Initializing all the Stores at 1675961739006 (+61721 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961739006Running coprocessor post-open hooks at 1675961739013 (+7 ms)Region opened successfully at 1675961739013 2023-02-09 16:55:39,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9., pid=94, masterSystemTime=1675961677278 2023-02-09 16:55:39,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:55:39,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:55:39,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=aa6a554ecfe82c532d26eba502eca4e9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:39,015 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961739015"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1675961739015"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675961739015"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675961739015"}]},"ts":"1675961739015"} 2023-02-09 16:55:39,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-09 16:55:39,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure aa6a554ecfe82c532d26eba502eca4e9, server=jenkins-hbase4.apache.org,43347,1675961647698 in 1 mins, 1.891 sec 2023-02-09 16:55:39,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-09 16:55:39,019 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, ASSIGN in 1 mins, 2.05 sec 2023-02-09 16:55:39,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.572 sec 2023-02-09 16:55:39,022 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675961676442","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62579,"client":"172.31.14.131:35024","queuetimems":0,"class":"HMaster"} 2023-02-09 16:55:39,022 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-09 16:55:40,916 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up 2023-02-09 16:55:40,917 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:40,917 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:40,918 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:40,919 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:40,919 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:40,919 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:40,919 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:40,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:40,921 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:40,922 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:40,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 16:55:40,922 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961740922"}]},"ts":"1675961740922"} 2023-02-09 16:55:40,923 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-09 16:55:40,928 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-09 16:55:40,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, UNASSIGN}] 2023-02-09 16:55:40,929 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, UNASSIGN 2023-02-09 16:55:40,930 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=aa6a554ecfe82c532d26eba502eca4e9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:40,930 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961740930"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675961740930"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675961740930"}]},"ts":"1675961740930"} 2023-02-09 16:55:40,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure aa6a554ecfe82c532d26eba502eca4e9, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:40,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, UNASSIGN 2023-02-09 16:55:40,933 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:40,933 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961740933"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961740933"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961740933"}]},"ts":"1675961740933"} 2023-02-09 16:55:40,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745}] 2023-02-09 16:55:41,022 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,023 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 16:55:41,083 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aa6a554ecfe82c532d26eba502eca4e9 2023-02-09 16:55:41,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aa6a554ecfe82c532d26eba502eca4e9, disabling compactions & flushes 2023-02-09 16:55:41,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:55:41,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:55:41,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. after waiting 0 ms 2023-02-09 16:55:41,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:55:41,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:41,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9. 2023-02-09 16:55:41,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aa6a554ecfe82c532d26eba502eca4e9: Waiting for close lock at 1675961741083Running coprocessor pre-close hooks at 1675961741083Disabling compacts and flushes for region at 1675961741083Disabling writes for close at 1675961741084 (+1 ms)Writing region close event to WAL at 1675961741084Running coprocessor post-close hooks at 1675961741084Closed at 1675961741085 (+1 ms) 2023-02-09 16:55:41,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:55:41,086 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 82b4da9176b3dc6382be1c3c41771bc2, disabling compactions & flushes 2023-02-09 16:55:41,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:55:41,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aa6a554ecfe82c532d26eba502eca4e9 2023-02-09 16:55:41,086 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:55:41,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. after waiting 0 ms 2023-02-09 16:55:41,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:55:41,087 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=aa6a554ecfe82c532d26eba502eca4e9, regionState=CLOSED 2023-02-09 16:55:41,087 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961741087"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675961741087"}]},"ts":"1675961741087"} 2023-02-09 16:55:41,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 16:55:41,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-09 16:55:41,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure aa6a554ecfe82c532d26eba502eca4e9, server=jenkins-hbase4.apache.org,43347,1675961647698 in 158 msec 2023-02-09 16:55:41,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:41,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2. 2023-02-09 16:55:41,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 82b4da9176b3dc6382be1c3c41771bc2: Waiting for close lock at 1675961741086Running coprocessor pre-close hooks at 1675961741086Disabling compacts and flushes for region at 1675961741086Disabling writes for close at 1675961741087 (+1 ms)Writing region close event to WAL at 1675961741087Running coprocessor post-close hooks at 1675961741092 (+5 ms)Closed at 1675961741092 2023-02-09 16:55:41,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9, UNASSIGN in 163 msec 2023-02-09 16:55:41,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:55:41,094 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=82b4da9176b3dc6382be1c3c41771bc2, regionState=CLOSED 2023-02-09 16:55:41,094 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675961741094"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961741094"}]},"ts":"1675961741094"} 2023-02-09 16:55:41,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-09 16:55:41,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 82b4da9176b3dc6382be1c3c41771bc2, server=jenkins-hbase4.apache.org,37073,1675961647745 in 161 msec 2023-02-09 16:55:41,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-09 16:55:41,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=82b4da9176b3dc6382be1c3c41771bc2, UNASSIGN in 168 msec 2023-02-09 16:55:41,099 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961741099"}]},"ts":"1675961741099"} 2023-02-09 16:55:41,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-09 16:55:41,101 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-09 16:55:41,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 183 msec 2023-02-09 16:55:41,223 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,224 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 16:55:41,225 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-09 16:55:41,225 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,225 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,225 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,228 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,228 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,228 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 16:55:41,233 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:55:41,235 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/recovered.edits] 2023-02-09 16:55:41,239 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/recovered.edits/7.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2/recovered.edits/7.seqid 2023-02-09 16:55:41,240 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/82b4da9176b3dc6382be1c3c41771bc2 2023-02-09 16:55:41,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 16:55:41,242 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,244 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-09 16:55:41,245 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-09 16:55:41,246 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,246 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-09 16:55:41,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961741246"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:41,247 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961741246"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:41,248 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-09 16:55:41,248 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 82b4da9176b3dc6382be1c3c41771bc2, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675961675935.82b4da9176b3dc6382be1c3c41771bc2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => aa6a554ecfe82c532d26eba502eca4e9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675961675935_0001.aa6a554ecfe82c532d26eba502eca4e9.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-09 16:55:41,248 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-09 16:55:41,248 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961741248"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:41,250 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-09 16:55:41,252 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:41,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 27 msec 2023-02-09 16:55:41,330 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,330 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 16:55:41,331 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-09 16:55:41,347 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 414), OpenFileDescriptor=655 (was 860), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 167), ProcessCount=168 (was 168), AvailableMemoryMB=2955 (was 3112) 2023-02-09 16:55:41,362 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=103, ProcessCount=168, AvailableMemoryMB=2955 2023-02-09 16:55:41,363 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:41,363 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,363 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,365 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:55:41,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:55:41,366 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:41,368 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:55:41,369 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:55:41,371 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,372 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 empty. 2023-02-09 16:55:41,372 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,372 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 16:55:41,385 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-09 16:55:41,386 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => ae42026b8fed495d9de02442023f9ac4, NAME => 'testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:55:41,394 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:41,394 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing ae42026b8fed495d9de02442023f9ac4, disabling compactions & flushes 2023-02-09 16:55:41,394 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,394 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,394 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. after waiting 0 ms 2023-02-09 16:55:41,394 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,394 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,394 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for ae42026b8fed495d9de02442023f9ac4: Waiting for close lock at 1675961741394Disabling compacts and flushes for region at 1675961741394Disabling writes for close at 1675961741394Writing region close event to WAL at 1675961741394Closed at 1675961741394 2023-02-09 16:55:41,397 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:55:41,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675961741398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961741398"}]},"ts":"1675961741398"} 2023-02-09 16:55:41,399 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:55:41,400 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:55:41,400 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961741400"}]},"ts":"1675961741400"} 2023-02-09 16:55:41,401 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-09 16:55:41,404 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:55:41,405 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:55:41,405 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:55:41,405 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:55:41,405 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:55:41,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, ASSIGN}] 2023-02-09 16:55:41,406 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, ASSIGN 2023-02-09 16:55:41,407 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:55:41,557 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:55:41,559 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ae42026b8fed495d9de02442023f9ac4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,559 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675961741559"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961741559"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961741559"}]},"ts":"1675961741559"} 2023-02-09 16:55:41,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure ae42026b8fed495d9de02442023f9ac4, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:41,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ae42026b8fed495d9de02442023f9ac4, NAME => 'testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:55:41,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:41,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:41,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,719 INFO [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,721 DEBUG [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1 2023-02-09 16:55:41,721 DEBUG [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1 2023-02-09 16:55:41,721 INFO [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae42026b8fed495d9de02442023f9ac4 columnFamilyName cf1 2023-02-09 16:55:41,722 INFO [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] regionserver.HStore(310): Store=ae42026b8fed495d9de02442023f9ac4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:41,722 INFO [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,723 DEBUG [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2 2023-02-09 16:55:41,723 DEBUG [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2 2023-02-09 16:55:41,724 INFO [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae42026b8fed495d9de02442023f9ac4 columnFamilyName cf2 2023-02-09 16:55:41,724 INFO [StoreOpener-ae42026b8fed495d9de02442023f9ac4-1] regionserver.HStore(310): Store=ae42026b8fed495d9de02442023f9ac4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:41,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:41,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:41,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:55:41,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ae42026b8fed495d9de02442023f9ac4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60432824, jitterRate=-0.09948074817657471}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:41,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ae42026b8fed495d9de02442023f9ac4: Running coprocessor pre-open hook at 1675961741718Writing region info on filesystem at 1675961741718Initializing all the Stores at 1675961741719 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961741719Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961741719Cleaning up temporary data from old regions at 1675961741726 (+7 ms)Cleaning up detritus from prior splits at 1675961741726Running coprocessor post-open hooks at 1675961741730 (+4 ms)Region opened successfully at 1675961741730 2023-02-09 16:55:41,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4., pid=104, masterSystemTime=1675961741713 2023-02-09 16:55:41,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ae42026b8fed495d9de02442023f9ac4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675961741733"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961741733"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961741733"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961741733"}]},"ts":"1675961741733"} 2023-02-09 16:55:41,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-09 16:55:41,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure ae42026b8fed495d9de02442023f9ac4, server=jenkins-hbase4.apache.org,33383,1675961647775 in 173 msec 2023-02-09 16:55:41,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-09 16:55:41,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, ASSIGN in 331 msec 2023-02-09 16:55:41,738 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:55:41,738 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961741738"}]},"ts":"1675961741738"} 2023-02-09 16:55:41,739 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-09 16:55:41,741 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:55:41,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 376 msec 2023-02-09 16:55:41,873 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:41,874 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:41,874 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:41,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 16:55:41,876 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 16:55:41,876 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:41,876 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 16:55:41,876 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 16:55:41,876 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,876 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,877 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:41,877 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:41,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:41,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:41,877 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:41,877 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:41,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:41,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:41,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:41,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:41,879 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:41,880 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:41,880 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:41,880 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,880 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. started... 2023-02-09 16:55:41,881 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:55:41,881 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,882 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,882 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,882 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:41,882 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,882 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:41,889 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/c6ad69d8655648b496d5316371751ef6 2023-02-09 16:55:41,901 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/556ceada20094f119107164a1c81c89f 2023-02-09 16:55:41,905 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/c6ad69d8655648b496d5316371751ef6 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/c6ad69d8655648b496d5316371751ef6 2023-02-09 16:55:41,908 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/c6ad69d8655648b496d5316371751ef6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:41,909 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/556ceada20094f119107164a1c81c89f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/556ceada20094f119107164a1c81c89f 2023-02-09 16:55:41,912 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/556ceada20094f119107164a1c81c89f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:41,913 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 32ms, sequenceid=6, compaction requested=false 2023-02-09 16:55:41,913 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-09 16:55:41,913 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:41,913 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:41,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:41,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:41,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:41,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:41,915 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,915 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:41,915 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,915 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:41,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:41,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:41,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:41,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:41,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 16:55:41,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@532ac7a4[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:41,917 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 16:55:41,917 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:41,918 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:41,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,918 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:41,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:41,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:41,920 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:41,920 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:41,920 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:41,921 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 16:55:41,924 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 16:55:41,924 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:41,924 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:41,924 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 16:55:41,924 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:41,925 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,925 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:41,925 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,925 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,925 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:41,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:41,925 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:41,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:41,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:41,926 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:41,927 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:41,928 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:41,928 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,929 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:41,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:41,931 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:41,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 16:55:41,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:41,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:41,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:41,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:41,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:41,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:41,931 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:41,932 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,032 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:42,032 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,032 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:42,042 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,042 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,043 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:42,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:42,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 16:55:42,044 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 16:55:42,044 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,044 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 16:55:42,044 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 16:55:42,045 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,045 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,047 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,047 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,047 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,047 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,048 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,048 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,049 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,049 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,049 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,049 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:42,049 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. started... 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,050 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,049 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,050 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,049 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:42,050 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,050 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,050 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,051 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,059 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/db2c310d9fb24adf8281312a815456aa 2023-02-09 16:55:42,069 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/e4089032ea36497d8d4c3833a4277f51 2023-02-09 16:55:42,072 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/db2c310d9fb24adf8281312a815456aa as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/db2c310d9fb24adf8281312a815456aa 2023-02-09 16:55:42,075 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/db2c310d9fb24adf8281312a815456aa, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:42,076 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/e4089032ea36497d8d4c3833a4277f51 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/e4089032ea36497d8d4c3833a4277f51 2023-02-09 16:55:42,080 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/e4089032ea36497d8d4c3833a4277f51, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:42,081 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 32ms, sequenceid=19, compaction requested=false 2023-02-09 16:55:42,081 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:42,081 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,081 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:42,081 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:42,081 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,081 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,081 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,083 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,083 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,084 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,084 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 16:55:42,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62a655d6[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:42,086 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 16:55:42,086 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,087 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,087 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:42,087 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,087 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,087 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:42,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,087 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,089 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,091 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,091 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,091 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 16:55:42,096 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 16:55:42,096 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:42,096 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:42,096 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 16:55:42,096 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,098 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,099 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,100 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,102 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,102 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,102 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,105 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,105 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,105 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testInvalidSplitKey because node does not exist (not an error) 2023-02-09 16:55:42,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 16:55:42,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:42,106 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:42,107 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:42,106 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,110 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,110 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,207 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:42,207 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,207 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:42,218 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,219 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,219 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:42,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:42,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 16:55:42,221 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 16:55:42,221 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,221 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 16:55:42,221 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 16:55:42,221 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,221 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,222 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,222 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,223 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,223 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,224 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,224 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,225 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:42,225 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:42,225 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. started... 2023-02-09 16:55:42,225 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,229 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,229 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,235 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/38514e6871c94e1492ceb531f0e3f8d6 2023-02-09 16:55:42,248 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/cc3ac6feae0e4bb2b7d4cea5b01189be 2023-02-09 16:55:42,252 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/38514e6871c94e1492ceb531f0e3f8d6 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/38514e6871c94e1492ceb531f0e3f8d6 2023-02-09 16:55:42,255 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/38514e6871c94e1492ceb531f0e3f8d6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:42,255 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/cc3ac6feae0e4bb2b7d4cea5b01189be as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/cc3ac6feae0e4bb2b7d4cea5b01189be 2023-02-09 16:55:42,259 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/cc3ac6feae0e4bb2b7d4cea5b01189be, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:42,259 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 34ms, sequenceid=32, compaction requested=true 2023-02-09 16:55:42,259 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:42,259 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,259 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:42,260 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:42,260 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,260 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,260 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,261 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,261 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,261 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,261 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 16:55:42,263 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7827769[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:42,263 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 16:55:42,263 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,264 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,264 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:42,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:42,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,265 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,265 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:42,267 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,267 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,267 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,267 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,267 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,267 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,268 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,268 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,268 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,268 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:42,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 16:55:42,271 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 16:55:42,271 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:42,271 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:42,271 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 16:55:42,271 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,273 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,272 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,273 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,273 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,275 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,275 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,275 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,282 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,282 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 16:55:42,282 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:42,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,282 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,283 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,383 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:42,383 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,384 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:42,395 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,395 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,396 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:42,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:42,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 16:55:42,397 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 16:55:42,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 16:55:42,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 16:55:42,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,398 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,399 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,400 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,400 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,401 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,401 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. started... 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,401 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:42,401 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,402 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,402 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,403 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,403 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,403 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,403 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,404 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,404 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,410 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/ba033c519edf4214b775d148e405316e 2023-02-09 16:55:42,420 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/7aec40d3d4d04d73aad8fe57bf6f7f78 2023-02-09 16:55:42,424 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/ba033c519edf4214b775d148e405316e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/ba033c519edf4214b775d148e405316e 2023-02-09 16:55:42,428 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/ba033c519edf4214b775d148e405316e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:42,429 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/7aec40d3d4d04d73aad8fe57bf6f7f78 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/7aec40d3d4d04d73aad8fe57bf6f7f78 2023-02-09 16:55:42,432 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/7aec40d3d4d04d73aad8fe57bf6f7f78, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:42,433 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 32ms, sequenceid=45, compaction requested=true 2023-02-09 16:55:42,433 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:42,433 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,433 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:42,433 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:42,433 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,434 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,434 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,436 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,436 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,436 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,436 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 16:55:42,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12382737[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:42,438 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 16:55:42,438 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:42,439 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:42,439 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:42,441 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,441 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,441 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,442 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,442 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,442 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,442 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:42,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 16:55:42,445 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 16:55:42,445 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:42,445 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:42,445 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 16:55:42,445 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,448 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,448 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,449 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,450 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,450 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,451 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,453 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 16:55:42,453 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,453 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:42,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,454 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,554 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:42,554 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,554 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:42,568 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,568 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,569 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:42,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:42,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 16:55:42,571 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 16:55:42,571 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,571 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 16:55:42,571 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 16:55:42,571 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,571 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,572 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,572 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,572 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,573 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,580 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,580 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,580 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,580 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,580 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,581 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,581 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,582 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,594 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,594 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,594 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,594 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. started... 2023-02-09 16:55:42,594 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:42,596 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,597 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,597 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,603 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/72f42d75de824aa19bf673823789a5e4 2023-02-09 16:55:42,615 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/59617438e4b0406d87b3acd094951d6f 2023-02-09 16:55:42,618 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/72f42d75de824aa19bf673823789a5e4 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/72f42d75de824aa19bf673823789a5e4 2023-02-09 16:55:42,621 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/72f42d75de824aa19bf673823789a5e4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:42,621 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/59617438e4b0406d87b3acd094951d6f as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/59617438e4b0406d87b3acd094951d6f 2023-02-09 16:55:42,624 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/59617438e4b0406d87b3acd094951d6f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:42,625 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 31ms, sequenceid=58, compaction requested=true 2023-02-09 16:55:42,625 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:42,625 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,625 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:42,625 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:42,625 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,625 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,625 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,627 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,627 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,627 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,627 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 16:55:42,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@172ac533[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:42,629 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 16:55:42,629 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:42,630 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,630 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,630 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:42,637 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,637 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,637 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,637 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,641 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:42,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 16:55:42,643 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 16:55:42,644 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:42,644 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:42,644 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 16:55:42,644 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,645 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,646 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,646 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,648 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,648 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,648 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,650 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 16:55:42,650 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,650 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,650 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,651 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,652 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,652 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,652 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,752 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:42,752 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,752 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:42,762 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,763 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,763 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:42,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:42,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 16:55:42,765 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 16:55:42,765 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,765 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 16:55:42,765 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 16:55:42,765 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,765 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,767 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,767 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,767 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,767 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,768 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,768 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 16:55:42,769 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,769 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,769 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:42,769 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:42,769 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,770 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. started... 2023-02-09 16:55:42,770 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,777 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/faf93ae7a7cf41348d0b3639ed79dbeb 2023-02-09 16:55:42,787 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/efab3bc186364ec78ef2759085ccac92 2023-02-09 16:55:42,791 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/faf93ae7a7cf41348d0b3639ed79dbeb as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/faf93ae7a7cf41348d0b3639ed79dbeb 2023-02-09 16:55:42,794 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/faf93ae7a7cf41348d0b3639ed79dbeb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:42,795 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/efab3bc186364ec78ef2759085ccac92 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/efab3bc186364ec78ef2759085ccac92 2023-02-09 16:55:42,798 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/efab3bc186364ec78ef2759085ccac92, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:42,799 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 29ms, sequenceid=71, compaction requested=true 2023-02-09 16:55:42,799 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:42,799 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:42,799 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:42,799 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:42,799 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,799 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 16:55:42,799 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 16:55:42,800 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,800 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,800 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,800 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:42,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 16:55:42,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47723ae5[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:42,802 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 16:55:42,802 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:42,804 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,804 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:42,804 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:42,806 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,806 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,806 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,807 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,807 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 16:55:42,807 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,807 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,807 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,807 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,807 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 16:55:42,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:42,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 16:55:42,810 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 16:55:42,810 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:42,810 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:42,810 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 16:55:42,810 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,812 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:42,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:42,814 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 16:55:42,815 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,815 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,817 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:42,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 16:55:42,818 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 16:55:42,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:42,817 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:42,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:42,818 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 16:55:42,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:42,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:42,919 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:42,919 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,919 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:42,926 INFO [Listener at localhost/39015] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,927 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.StoreUtils(129): cannot split hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/db2c310d9fb24adf8281312a815456aa because midkey is the same as first or last row 2023-02-09 16:55:42,927 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.StoreUtils(129): cannot split hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/e4089032ea36497d8d4c3833a4277f51 because midkey is the same as first or last row 2023-02-09 16:55:42,927 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegion(2744): Flushing ae42026b8fed495d9de02442023f9ac4 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:55:42,936 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/6083a49f9f614afebbbd62a29b41ae26 2023-02-09 16:55:42,946 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/674b665a0ebe4947be7e3606162e36a7 2023-02-09 16:55:42,950 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf1/6083a49f9f614afebbbd62a29b41ae26 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/6083a49f9f614afebbbd62a29b41ae26 2023-02-09 16:55:42,953 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/6083a49f9f614afebbbd62a29b41ae26, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:42,954 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/.tmp/cf2/674b665a0ebe4947be7e3606162e36a7 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/674b665a0ebe4947be7e3606162e36a7 2023-02-09 16:55:42,956 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/674b665a0ebe4947be7e3606162e36a7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:42,957 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ae42026b8fed495d9de02442023f9ac4 in 30ms, sequenceid=82, compaction requested=true 2023-02-09 16:55:42,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.HRegion(2445): Flush status journal for ae42026b8fed495d9de02442023f9ac4: 2023-02-09 16:55:42,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.StoreUtils(129): cannot split hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/db2c310d9fb24adf8281312a815456aa because midkey is the same as first or last row 2023-02-09 16:55:42,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33383] regionserver.StoreUtils(129): cannot split hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/e4089032ea36497d8d4c3833a4277f51 because midkey is the same as first or last row 2023-02-09 16:55:42,959 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,959 DEBUG [Listener at localhost/39015] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-09 16:55:42,959 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:42,959 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,959 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,960 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,960 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,960 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-09 16:55:42,961 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-09 16:55:42,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:42,963 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:42,963 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:42,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 16:55:42,964 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961742963"}]},"ts":"1675961742963"} 2023-02-09 16:55:42,964 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-09 16:55:42,967 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-09 16:55:42,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, UNASSIGN}] 2023-02-09 16:55:42,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, UNASSIGN 2023-02-09 16:55:42,968 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ae42026b8fed495d9de02442023f9ac4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:42,968 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675961742968"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961742968"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961742968"}]},"ts":"1675961742968"} 2023-02-09 16:55:42,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure ae42026b8fed495d9de02442023f9ac4, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:43,064 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,064 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 16:55:43,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:43,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ae42026b8fed495d9de02442023f9ac4, disabling compactions & flushes 2023-02-09 16:55:43,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:43,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:43,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. after waiting 0 ms 2023-02-09 16:55:43,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:43,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:55:43,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:43,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4. 2023-02-09 16:55:43,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ae42026b8fed495d9de02442023f9ac4: Waiting for close lock at 1675961743121Running coprocessor pre-close hooks at 1675961743121Disabling compacts and flushes for region at 1675961743121Disabling writes for close at 1675961743122 (+1 ms)Writing region close event to WAL at 1675961743122Running coprocessor post-close hooks at 1675961743126 (+4 ms)Closed at 1675961743126 2023-02-09 16:55:43,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:43,128 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ae42026b8fed495d9de02442023f9ac4, regionState=CLOSED 2023-02-09 16:55:43,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675961743128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961743128"}]},"ts":"1675961743128"} 2023-02-09 16:55:43,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-09 16:55:43,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure ae42026b8fed495d9de02442023f9ac4, server=jenkins-hbase4.apache.org,33383,1675961647775 in 160 msec 2023-02-09 16:55:43,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-09 16:55:43,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ae42026b8fed495d9de02442023f9ac4, UNASSIGN in 164 msec 2023-02-09 16:55:43,132 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961743132"}]},"ts":"1675961743132"} 2023-02-09 16:55:43,133 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-09 16:55:43,136 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-09 16:55:43,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-02-09 16:55:43,265 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,265 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 16:55:43,266 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-09 16:55:43,266 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,266 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-09 16:55:43,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:43,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:43,269 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,269 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,269 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:43,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 16:55:43,272 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:43,273 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/recovered.edits] 2023-02-09 16:55:43,276 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/38514e6871c94e1492ceb531f0e3f8d6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/38514e6871c94e1492ceb531f0e3f8d6 2023-02-09 16:55:43,277 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/6083a49f9f614afebbbd62a29b41ae26 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/6083a49f9f614afebbbd62a29b41ae26 2023-02-09 16:55:43,278 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/72f42d75de824aa19bf673823789a5e4 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/72f42d75de824aa19bf673823789a5e4 2023-02-09 16:55:43,279 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/ba033c519edf4214b775d148e405316e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/ba033c519edf4214b775d148e405316e 2023-02-09 16:55:43,280 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/c6ad69d8655648b496d5316371751ef6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/c6ad69d8655648b496d5316371751ef6 2023-02-09 16:55:43,282 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/db2c310d9fb24adf8281312a815456aa to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/db2c310d9fb24adf8281312a815456aa 2023-02-09 16:55:43,283 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/faf93ae7a7cf41348d0b3639ed79dbeb to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf1/faf93ae7a7cf41348d0b3639ed79dbeb 2023-02-09 16:55:43,285 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/556ceada20094f119107164a1c81c89f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/556ceada20094f119107164a1c81c89f 2023-02-09 16:55:43,286 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/59617438e4b0406d87b3acd094951d6f to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/59617438e4b0406d87b3acd094951d6f 2023-02-09 16:55:43,287 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/674b665a0ebe4947be7e3606162e36a7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/674b665a0ebe4947be7e3606162e36a7 2023-02-09 16:55:43,288 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/7aec40d3d4d04d73aad8fe57bf6f7f78 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/7aec40d3d4d04d73aad8fe57bf6f7f78 2023-02-09 16:55:43,289 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/cc3ac6feae0e4bb2b7d4cea5b01189be to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/cc3ac6feae0e4bb2b7d4cea5b01189be 2023-02-09 16:55:43,290 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/e4089032ea36497d8d4c3833a4277f51 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/e4089032ea36497d8d4c3833a4277f51 2023-02-09 16:55:43,291 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/efab3bc186364ec78ef2759085ccac92 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/cf2/efab3bc186364ec78ef2759085ccac92 2023-02-09 16:55:43,293 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/recovered.edits/85.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4/recovered.edits/85.seqid 2023-02-09 16:55:43,293 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testInvalidSplitKey/ae42026b8fed495d9de02442023f9ac4 2023-02-09 16:55:43,293 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 16:55:43,295 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:43,296 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-09 16:55:43,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-09 16:55:43,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:43,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-09 16:55:43,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961743301"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:43,302 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 16:55:43,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ae42026b8fed495d9de02442023f9ac4, NAME => 'testInvalidSplitKey,,1675961741366.ae42026b8fed495d9de02442023f9ac4.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 16:55:43,302 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-09 16:55:43,302 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961743302"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:43,303 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-09 16:55:43,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:43,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 39 msec 2023-02-09 16:55:43,370 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,370 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 16:55:43,371 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-09 16:55:43,384 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=367 (was 348) - Thread LEAK? -, OpenFileDescriptor=742 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 103), ProcessCount=168 (was 168), AvailableMemoryMB=2950 (was 2955) 2023-02-09 16:55:43,397 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=367, OpenFileDescriptor=742, MaxFileDescriptor=60000, SystemLoadAverage=103, ProcessCount=168, AvailableMemoryMB=2949 2023-02-09 16:55:43,397 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:43,398 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,398 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,399 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:55:43,400 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:55:43,401 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:43,403 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:55:43,403 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:55:43,405 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,405 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 empty. 2023-02-09 16:55:43,405 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,405 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 16:55:43,414 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 16:55:43,415 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8dd6d0e199e1c44e37d692efb64f3fa5, NAME => 'testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:55:43,422 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:43,422 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 8dd6d0e199e1c44e37d692efb64f3fa5, disabling compactions & flushes 2023-02-09 16:55:43,422 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,422 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,423 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. after waiting 0 ms 2023-02-09 16:55:43,423 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,423 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,423 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 8dd6d0e199e1c44e37d692efb64f3fa5: Waiting for close lock at 1675961743422Disabling compacts and flushes for region at 1675961743422Disabling writes for close at 1675961743423 (+1 ms)Writing region close event to WAL at 1675961743423Closed at 1675961743423 2023-02-09 16:55:43,424 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:55:43,425 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961743425"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961743425"}]},"ts":"1675961743425"} 2023-02-09 16:55:43,426 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:55:43,426 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:55:43,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961743427"}]},"ts":"1675961743427"} 2023-02-09 16:55:43,427 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 16:55:43,431 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:55:43,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:55:43,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:55:43,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:55:43,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:55:43,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, ASSIGN}] 2023-02-09 16:55:43,434 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, ASSIGN 2023-02-09 16:55:43,434 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:55:43,584 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:55:43,585 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961743585"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961743585"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961743585"}]},"ts":"1675961743585"} 2023-02-09 16:55:43,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:43,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8dd6d0e199e1c44e37d692efb64f3fa5, NAME => 'testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:55:43,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:43,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:43,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,742 INFO [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,744 DEBUG [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1 2023-02-09 16:55:43,744 DEBUG [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1 2023-02-09 16:55:43,744 INFO [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8dd6d0e199e1c44e37d692efb64f3fa5 columnFamilyName cf1 2023-02-09 16:55:43,744 INFO [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] regionserver.HStore(310): Store=8dd6d0e199e1c44e37d692efb64f3fa5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:43,745 INFO [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,745 DEBUG [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2 2023-02-09 16:55:43,745 DEBUG [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2 2023-02-09 16:55:43,746 INFO [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8dd6d0e199e1c44e37d692efb64f3fa5 columnFamilyName cf2 2023-02-09 16:55:43,746 INFO [StoreOpener-8dd6d0e199e1c44e37d692efb64f3fa5-1] regionserver.HStore(310): Store=8dd6d0e199e1c44e37d692efb64f3fa5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:43,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:43,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:43,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:55:43,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8dd6d0e199e1c44e37d692efb64f3fa5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62298748, jitterRate=-0.07167631387710571}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:43,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8dd6d0e199e1c44e37d692efb64f3fa5: Running coprocessor pre-open hook at 1675961743742Writing region info on filesystem at 1675961743742Initializing all the Stores at 1675961743742Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961743742Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961743742Cleaning up temporary data from old regions at 1675961743747 (+5 ms)Cleaning up detritus from prior splits at 1675961743748 (+1 ms)Running coprocessor post-open hooks at 1675961743751 (+3 ms)Region opened successfully at 1675961743751 2023-02-09 16:55:43,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5., pid=111, masterSystemTime=1675961743738 2023-02-09 16:55:43,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,753 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961743753"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961743753"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961743753"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961743753"}]},"ts":"1675961743753"} 2023-02-09 16:55:43,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-09 16:55:43,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 in 167 msec 2023-02-09 16:55:43,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-09 16:55:43,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, ASSIGN in 323 msec 2023-02-09 16:55:43,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:55:43,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961743757"}]},"ts":"1675961743757"} 2023-02-09 16:55:43,758 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 16:55:43,760 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:55:43,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 359 msec 2023-02-09 16:55:43,908 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:43,908 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:43,908 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:43,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 16:55:43,910 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:43,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:43,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 16:55:43,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:55:43,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,910 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,912 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:43,912 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:43,912 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:43,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:43,912 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:43,912 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,912 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:43,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:43,913 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:43,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:43,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:43,914 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:43,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:43,914 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:43,914 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,914 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. started... 2023-02-09 16:55:43,916 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:55:43,917 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,917 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,917 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,917 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,918 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:43,925 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/bc9546020eb14893a2abac5d2609a3d8 2023-02-09 16:55:43,935 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/56164a8006714149a4b1bbf6385e72d0 2023-02-09 16:55:43,939 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/bc9546020eb14893a2abac5d2609a3d8 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 2023-02-09 16:55:43,942 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:43,943 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/56164a8006714149a4b1bbf6385e72d0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 2023-02-09 16:55:43,946 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:43,947 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 31ms, sequenceid=6, compaction requested=false 2023-02-09 16:55:43,947 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:43,948 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8dd6d0e199e1c44e37d692efb64f3fa5: 2023-02-09 16:55:43,948 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:43,948 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:43,948 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:43,948 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,948 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:43,948 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:43,949 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,949 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:43,950 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,950 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 16:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53df245d[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:43,951 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:55:43,952 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:43,953 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:43,953 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:43,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:43,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:43,953 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:43,955 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:43,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:43,957 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:43,957 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:43,957 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:43,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:55:43,960 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 16:55:43,960 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:43,960 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:43,960 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 16:55:43,960 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:43,961 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,961 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:43,961 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,961 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:43,962 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:43,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:43,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:43,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:43,963 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,963 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:43,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:43,964 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:43,965 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,965 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:43,968 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:43,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:43,968 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:43,969 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,968 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:43,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:43,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:43,969 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:43,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:43,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,070 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:44,070 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,070 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:44,080 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,081 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,081 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:44,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:44,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 16:55:44,083 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,083 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,083 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 16:55:44,083 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:55:44,083 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,083 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,084 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,084 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,084 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,084 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,084 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,084 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,086 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:44,087 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:44,087 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,087 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. started... 2023-02-09 16:55:44,087 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,088 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,095 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/58010a1dedb54abca0958a6d858c75c6 2023-02-09 16:55:44,111 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/cc64d87b6ee846f3b42dffc52306c6a3 2023-02-09 16:55:44,115 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/58010a1dedb54abca0958a6d858c75c6 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 2023-02-09 16:55:44,119 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:44,120 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/cc64d87b6ee846f3b42dffc52306c6a3 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 2023-02-09 16:55:44,123 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:44,124 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 37ms, sequenceid=19, compaction requested=false 2023-02-09 16:55:44,124 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8dd6d0e199e1c44e37d692efb64f3fa5: 2023-02-09 16:55:44,124 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,124 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,132 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,132 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,133 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,133 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 16:55:44,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8db08e[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:44,134 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:55:44,135 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:44,136 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:44,136 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,137 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,139 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,139 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,139 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,139 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,139 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,139 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,140 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:44,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:55:44,142 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 16:55:44,143 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:44,143 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:44,143 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,143 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:44,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,144 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,145 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,145 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,145 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,146 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,147 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,147 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,148 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,153 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:44,152 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,152 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:44,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,153 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,154 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,154 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,154 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,154 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,254 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:44,254 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,254 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:44,264 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,264 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,265 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:44,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:44,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 16:55:44,266 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,266 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,266 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 16:55:44,266 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:55:44,267 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,267 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,268 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,268 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,268 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,269 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:44,270 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,270 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,271 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. started... 2023-02-09 16:55:44,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,271 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:44,277 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,277 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,277 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,278 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,279 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,279 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,286 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/d67176a92656446c90e1c07119e978b1 2023-02-09 16:55:44,297 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/6825808da9a54b4fb43d0d81e5c4b002 2023-02-09 16:55:44,302 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/d67176a92656446c90e1c07119e978b1 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 2023-02-09 16:55:44,306 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:44,307 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/6825808da9a54b4fb43d0d81e5c4b002 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 2023-02-09 16:55:44,311 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:44,312 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 41ms, sequenceid=32, compaction requested=true 2023-02-09 16:55:44,312 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8dd6d0e199e1c44e37d692efb64f3fa5: 2023-02-09 16:55:44,312 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,312 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:44,312 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:44,312 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,312 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,312 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,314 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,314 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,314 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,314 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 16:55:44,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@414feef1[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:44,317 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:55:44,317 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,318 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,318 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:44,318 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,322 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,322 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,322 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,322 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,323 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,323 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,323 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:44,327 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:55:44,327 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 16:55:44,327 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:44,327 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:44,327 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,327 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,329 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,330 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,330 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,332 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,333 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,333 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,340 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,340 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,341 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:44,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,340 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,341 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,341 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,341 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:44,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,341 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:44,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,341 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,341 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,342 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,441 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:44,441 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,442 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:44,452 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,452 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,452 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:44,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:44,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 16:55:44,454 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 16:55:44,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:55:44,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,454 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,455 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,456 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,456 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,455 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,456 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,456 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:44,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,457 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,457 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,457 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,458 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:44,458 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,458 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. started... 2023-02-09 16:55:44,458 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,458 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:44,460 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,469 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/88fffc7261934215858fe2d39d84d693 2023-02-09 16:55:44,487 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/227c5f0fc02d456dbec680182669da5b 2023-02-09 16:55:44,492 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/88fffc7261934215858fe2d39d84d693 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 2023-02-09 16:55:44,497 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:44,498 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/227c5f0fc02d456dbec680182669da5b as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b 2023-02-09 16:55:44,502 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:44,503 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 45ms, sequenceid=45, compaction requested=true 2023-02-09 16:55:44,503 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8dd6d0e199e1c44e37d692efb64f3fa5: 2023-02-09 16:55:44,503 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,503 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:44,503 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:44,503 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,505 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,505 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,506 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,506 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 16:55:44,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17bc412f[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:44,508 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:55:44,508 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,509 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,513 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,513 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,513 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,513 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:55:44,516 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 16:55:44,516 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:44,516 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:44,516 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,516 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:44,517 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,517 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,517 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,518 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,517 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,518 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,518 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,519 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,521 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,521 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,521 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,524 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,524 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,525 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:44,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,524 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:44,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,525 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:44,577 ERROR [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid 762ffd31-8114-414e-bb4b-20a4d577f100) service to localhost/127.0.0.1:44699 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:55:44,625 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:44,625 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,626 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:44,636 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,636 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,637 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:44,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:44,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 16:55:44,638 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,638 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,638 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 16:55:44,638 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:55:44,639 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,639 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,640 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,640 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,640 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,640 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,641 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,641 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:44,643 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,643 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. started... 2023-02-09 16:55:44,643 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,652 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/2411c4999cbf48608455a738a11a2622 2023-02-09 16:55:44,666 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/ca993c9199d14e1faf1b00c089b16139 2023-02-09 16:55:44,670 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/2411c4999cbf48608455a738a11a2622 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 2023-02-09 16:55:44,673 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:44,673 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/ca993c9199d14e1faf1b00c089b16139 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 2023-02-09 16:55:44,676 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:44,677 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 34ms, sequenceid=58, compaction requested=true 2023-02-09 16:55:44,677 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8dd6d0e199e1c44e37d692efb64f3fa5: 2023-02-09 16:55:44,677 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,678 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,681 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 16:55:44,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3deb6249[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:44,683 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:55:44,683 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,684 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:44,685 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:44,686 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,686 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,686 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,691 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,691 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:44,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:55:44,694 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 16:55:44,694 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:44,694 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:44,694 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,694 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:44,695 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,695 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,696 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,695 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,696 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,696 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,697 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,698 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,698 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,699 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,701 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,701 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:44,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,802 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:44,802 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,802 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:44,814 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,814 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,815 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:44,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:44,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 16:55:44,816 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,816 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,816 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 16:55:44,816 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 16:55:44,817 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,817 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,819 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,819 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,819 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,819 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,819 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,819 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:44,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,821 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,821 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 16:55:44,821 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:44,821 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 16:55:44,822 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:44,822 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:44,822 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,822 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. started... 2023-02-09 16:55:44,823 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:44,823 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,824 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,824 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,824 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,825 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,825 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,833 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/a331a6db0abf4663877c1cbd095967e0 2023-02-09 16:55:44,849 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/1130ee0cefd849bf8b70070b44127709 2023-02-09 16:55:44,853 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/a331a6db0abf4663877c1cbd095967e0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 2023-02-09 16:55:44,857 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:44,858 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/1130ee0cefd849bf8b70070b44127709 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 2023-02-09 16:55:44,862 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:44,863 INFO [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 41ms, sequenceid=71, compaction requested=true 2023-02-09 16:55:44,863 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8dd6d0e199e1c44e37d692efb64f3fa5: 2023-02-09 16:55:44,863 DEBUG [rs(jenkins-hbase4.apache.org,33383,1675961647775)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:44,863 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:44,863 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:44,863 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,863 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 16:55:44,863 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 16:55:44,865 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,865 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,865 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 16:55:44,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@508facba[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:44,868 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 16:55:44,868 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,869 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,869 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:44,870 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,870 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,872 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 16:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,873 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 16:55:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33383,1675961647775': 2023-02-09 16:55:44,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 16:55:44,878 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 16:55:44,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:44,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:44,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 16:55:44,878 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:44,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:44,881 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,881 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,881 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:44,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,883 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:44,883 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 16:55:44,884 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:44,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 16:55:44,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:44,892 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:44,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:44,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:44,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,892 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:44,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,893 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 16:55:44,993 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:44,993 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:44,993 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:44,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:45,253 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 16:55:45,254 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:45,255 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:45,257 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:45,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:45,505 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,505 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:45,505 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:45,505 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:45,505 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-09 16:55:45,506 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:45,506 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:45,506 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:45,506 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:45,506 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:45,508 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 16:55:45,508 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:45,508 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:45,509 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:45,509 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:45,509 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:45,509 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:45,510 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:45,510 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:45,510 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:45,510 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:45,511 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:45,511 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:45,511 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:45,511 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:45,512 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:45,512 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:45,512 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:45,512 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:45,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:45,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:45,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:45,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:45,513 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:45,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:45,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:45,514 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:45,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,515 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:45,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:45,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:45,516 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:45,517 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,517 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:45,517 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,517 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,517 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,517 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 16:55:45,518 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:45,518 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:45,519 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961775519 2023-02-09 16:55:45,520 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:45,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,522 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:45,522 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:45,522 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:45,522 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961805522 2023-02-09 16:55:45,523 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961865523 2023-02-09 16:55:45,523 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:55:45,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:45,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:45,524 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:45,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:45,773 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:45,773 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:45,774 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:45,774 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:45,775 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 16:55:45,776 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:45,776 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:45,776 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:45,776 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:45,776 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:45,777 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:45,778 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,779 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:45,780 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:45,781 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,781 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:45,781 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,781 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,781 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,781 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 16:55:45,781 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:45,782 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:45,782 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961775782 2023-02-09 16:55:45,782 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:45,784 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,786 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:45,787 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:45,787 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:45,787 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961805787 2023-02-09 16:55:45,787 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961865787 2023-02-09 16:55:45,787 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:55:45,787 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:45,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN}] 2023-02-09 16:55:45,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:45,789 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:45,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:45,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:46,037 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,037 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:46,037 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:46,037 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:46,037 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:46,038 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:46,038 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:46,038 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:46,038 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:46,038 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:46,040 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 16:55:46,040 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:46,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:46,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:46,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:46,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,041 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:46,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:46,042 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,043 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,043 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,043 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:46,043 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:46,043 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,043 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,044 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,044 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:46,044 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,044 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,044 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,045 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:46,045 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,045 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,045 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,046 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,047 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:46,047 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,047 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:46,047 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,047 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,047 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,047 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 16:55:46,048 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:46,048 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:46,051 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961776051 2023-02-09 16:55:46,053 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,053 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:46,055 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,055 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,056 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:46,056 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961806056 2023-02-09 16:55:46,056 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961866056 2023-02-09 16:55:46,056 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:55:46,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN}] 2023-02-09 16:55:46,056 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:46,059 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:46,059 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:46,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961746059"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961746059"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961746059"}]},"ts":"1675961746059"} 2023-02-09 16:55:46,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:46,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8dd6d0e199e1c44e37d692efb64f3fa5, disabling compactions & flushes 2023-02-09 16:55:46,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:46,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:46,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. after waiting 0 ms 2023-02-09 16:55:46,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:46,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8dd6d0e199e1c44e37d692efb64f3fa5 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:55:46,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/21c2bd5f19714e4d8cee65684f541aa4 2023-02-09 16:55:46,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/77d08aca744c4a2e9f9443da7b73891a 2023-02-09 16:55:46,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf1/21c2bd5f19714e4d8cee65684f541aa4 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 2023-02-09 16:55:46,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:46,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/.tmp/cf2/77d08aca744c4a2e9f9443da7b73891a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a 2023-02-09 16:55:46,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:46,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 40ms, sequenceid=82, compaction requested=true 2023-02-09 16:55:46,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:55:46,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:46,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. 2023-02-09 16:55:46,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8dd6d0e199e1c44e37d692efb64f3fa5: Waiting for close lock at 1675961746216Running coprocessor pre-close hooks at 1675961746216Disabling compacts and flushes for region at 1675961746216Disabling writes for close at 1675961746217 (+1 ms)Obtaining lock to block concurrent updates at 1675961746217Preparing flush snapshotting stores in 8dd6d0e199e1c44e37d692efb64f3fa5 at 1675961746217Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675961746217Flushing stores of testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. at 1675961746217Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf1: creating writer at 1675961746218 (+1 ms)Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf1: appending metadata at 1675961746220 (+2 ms)Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf1: closing flushed file at 1675961746220Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf2: creating writer at 1675961746230 (+10 ms)Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf2: appending metadata at 1675961746232 (+2 ms)Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf2: closing flushed file at 1675961746232Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf1: reopening flushed file at 1675961746245 (+13 ms)Flushing 8dd6d0e199e1c44e37d692efb64f3fa5/cf2: reopening flushed file at 1675961746252 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8dd6d0e199e1c44e37d692efb64f3fa5 in 40ms, sequenceid=82, compaction requested=true at 1675961746257 (+5 ms)Writing region close event to WAL at 1675961746258 (+1 ms)Running coprocessor post-close hooks at 1675961746261 (+3 ms)Closed at 1675961746262 (+1 ms) 2023-02-09 16:55:46,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,266 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED 2023-02-09 16:55:46,266 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961746266"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961746266"}]},"ts":"1675961746266"} 2023-02-09 16:55:46,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 16:55:46,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 in 203 msec 2023-02-09 16:55:46,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 16:55:46,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN in 213 msec 2023-02-09 16:55:46,272 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:46,272 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:46,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:46,306 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,306 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:46,306 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:46,306 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:46,306 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:46,307 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:46,307 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:46,307 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:46,307 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:46,307 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:46,309 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 16:55:46,309 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:46,309 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:46,309 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:46,309 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,310 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:46,311 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,312 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:46,313 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,313 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,313 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,313 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,313 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:46,313 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,314 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,314 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,314 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,314 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,314 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:46,315 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,315 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:46,315 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:46,315 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:46,315 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,315 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,315 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,315 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:46,315 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:46,315 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 16:55:46,316 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:46,316 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:46,317 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961776317 2023-02-09 16:55:46,318 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:46,318 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,320 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,321 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,321 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:46,321 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961806321 2023-02-09 16:55:46,321 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961866321 2023-02-09 16:55:46,321 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:55:46,323 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:46,331 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=8dd6d0e199e1c44e37d692efb64f3fa5, threads=14 2023-02-09 16:55:46,331 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,332 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,333 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,333 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,333 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,333 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,333 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,428 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,431 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,433 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,434 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,435 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,441 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,442 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,445 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,445 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,445 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,446 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,448 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,452 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,452 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,452 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 8dd6d0e199e1c44e37d692efb64f3fa5 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:55:46,479 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:46,479 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:46,480 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:46,571 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,571 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:46,571 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:46,571 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:46,572 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:46,572 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:46,572 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:46,572 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:46,572 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:46,573 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:46,575 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 16:55:46,575 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:46,575 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:46,576 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:46,576 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:46,576 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:46,576 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:46,577 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,577 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:46,577 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,577 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,578 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,578 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:46,578 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,578 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,579 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,579 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:46,579 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:46,579 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,579 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,580 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,580 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:46,580 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,580 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,581 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,581 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:46,581 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,581 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,581 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,582 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,582 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:46,582 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,582 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:46,583 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:46,584 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:46,584 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,584 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,584 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,584 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:46,584 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:46,584 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 16:55:46,585 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:46,585 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:46,587 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961776587 2023-02-09 16:55:46,589 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,589 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:46,591 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,591 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,592 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:46,592 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961806592 2023-02-09 16:55:46,592 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961866592 2023-02-09 16:55:46,592 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 16:55:46,602 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=8dd6d0e199e1c44e37d692efb64f3fa5, threads=14 2023-02-09 16:55:46,602 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,603 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,603 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,603 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,603 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,603 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,603 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,604 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,604 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,608 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,608 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,608 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,608 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,609 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,623 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:55:46,767 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,768 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,769 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,771 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,774 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,777 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,778 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,784 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,787 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,789 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,791 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,792 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,792 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,792 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a for region: 8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:46,792 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 8dd6d0e199e1c44e37d692efb64f3fa5 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:55:46,819 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:46,823 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:55:46,825 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:55:46,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:46,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:46,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:46,842 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,842 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:46,842 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:46,842 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:46,842 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:46,842 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:46,842 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:46,843 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:46,843 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:46,843 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:46,844 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 16:55:46,844 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:46,844 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:46,844 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:46,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:46,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:46,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:46,845 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:46,846 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,847 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:46,847 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:46,847 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,847 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:46,847 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,847 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:46,848 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,849 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:46,850 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,850 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,850 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:46,850 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:46,850 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 16:55:46,851 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:46,851 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:46,852 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961776852 2023-02-09 16:55:46,853 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:46,853 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,855 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,855 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:46,856 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:46,856 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961806856 2023-02-09 16:55:46,856 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961866856 2023-02-09 16:55:46,856 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:55:46,858 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:46,860 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:46,860 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:46,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:46,860 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:47,106 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,106 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:47,106 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:47,106 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:47,107 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:47,107 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:47,107 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:47,107 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:47,107 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:47,108 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:47,109 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 16:55:47,110 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:47,110 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:47,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:47,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:47,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:47,110 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,111 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,112 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,113 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:47,114 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,115 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:47,115 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:47,115 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:47,115 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,115 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,115 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,115 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,115 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,115 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 16:55:47,116 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:47,116 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:47,117 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961777117 2023-02-09 16:55:47,117 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:47,118 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,120 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,120 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,121 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:47,121 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961807121 2023-02-09 16:55:47,121 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961867121 2023-02-09 16:55:47,121 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:55:47,121 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:47,123 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961747123"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675961747123"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675961747123"}]},"ts":"1675961747123"} 2023-02-09 16:55:47,123 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961747123"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961747123"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961747123"}]},"ts":"1675961747123"} 2023-02-09 16:55:47,123 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961747123"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961747123"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961747123"}]},"ts":"1675961747123"} 2023-02-09 16:55:47,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:47,124 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:47,124 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:47,371 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,371 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:47,371 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:47,371 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:47,371 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:47,371 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:47,372 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:47,372 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:47,372 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:47,372 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:47,373 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 16:55:47,374 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:47,374 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:47,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:47,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:47,374 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:47,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:47,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:47,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,375 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:47,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,376 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,377 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:47,377 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:47,377 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,377 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,377 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,378 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:47,378 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,378 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,378 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,379 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:47,379 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,379 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,379 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,380 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,380 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:47,380 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,380 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,380 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,381 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,381 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,381 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:47,381 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,381 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:47,382 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:47,382 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:47,382 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,382 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,382 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,382 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,382 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,382 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-09 16:55:47,388 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:47,391 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:47,393 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961777393 2023-02-09 16:55:47,393 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:47,395 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,398 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,398 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,398 INFO [Listener at localhost/39015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33383,1675961647775, table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:47,398 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=9fa51a9d7502e93206005ef7befc7fc1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 16:55:47,399 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=dee274540666147336eaecbeea082487, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 16:55:47,399 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:47,399 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961807399 2023-02-09 16:55:47,399 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961867399 2023-02-09 16:55:47,399 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-09 16:55:47,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675961747399"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675961747399"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675961747399"}]},"ts":"1675961747399"} 2023-02-09 16:55:47,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961747399"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961747399"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961747399"}]},"ts":"1675961747399"} 2023-02-09 16:55:47,400 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961747399"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961747399"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961747399"}]},"ts":"1675961747399"} 2023-02-09 16:55:47,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:47,403 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:47,403 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:47,404 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:47,577 ERROR [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid e1d56305-25f8-42c9-ab95-be1e251b8441) service to localhost/127.0.0.1:44699 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:55:47,649 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,649 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:47,649 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:47,649 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:47,650 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:47,650 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:47,650 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:47,650 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:47,650 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:47,650 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:47,651 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-09 16:55:47,652 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:47,652 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:47,652 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:47,652 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:47,652 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:47,652 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:47,653 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,653 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:47,653 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,653 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,654 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,654 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:47,654 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,654 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,654 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,654 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,655 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,656 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:47,657 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:47,657 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,657 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,657 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,658 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,658 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,658 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 16:55:47,658 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:47,659 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:47,659 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961777659 2023-02-09 16:55:47,660 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:47,660 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,662 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,663 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,663 INFO [Listener at localhost/39015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33383,1675961647775, table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:47,663 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=9fa51a9d7502e93206005ef7befc7fc1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 16:55:47,663 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=dee274540666147336eaecbeea082487, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 16:55:47,663 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:47,663 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961807663 2023-02-09 16:55:47,663 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961867663 2023-02-09 16:55:47,663 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:55:47,663 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:47,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN}] 2023-02-09 16:55:47,665 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:47,665 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:47,665 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:47,913 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:47,914 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:47,914 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:47,915 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:47,916 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-09 16:55:47,916 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:47,916 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:47,917 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:47,917 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:47,917 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:47,918 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:47,918 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,918 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:47,918 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,918 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:47,919 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,919 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:47,919 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,919 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:47,919 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,920 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:47,920 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:47,920 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,920 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:47,921 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,921 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:47,921 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,921 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:47,921 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,921 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:47,922 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,922 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:47,922 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,922 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,923 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:47,923 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,923 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:47,923 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,923 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,923 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:47,924 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,924 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,924 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,924 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:47,925 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-09 16:55:47,925 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:47,925 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:47,926 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961777926 2023-02-09 16:55:47,927 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:47,927 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:47,929 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,929 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:47,929 INFO [Listener at localhost/39015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33383,1675961647775, table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:47,930 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=9fa51a9d7502e93206005ef7befc7fc1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 16:55:47,930 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=dee274540666147336eaecbeea082487, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 16:55:47,930 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:47,930 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961807930 2023-02-09 16:55:47,930 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961867930 2023-02-09 16:55:47,930 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 16:55:47,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN}] 2023-02-09 16:55:47,930 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:47,932 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN 2023-02-09 16:55:47,932 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:55:47,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN 2023-02-09 16:55:47,933 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33383,1675961647775; forceNewPlan=false, retain=false 2023-02-09 16:55:48,083 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:55:48,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=9fa51a9d7502e93206005ef7befc7fc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,085 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961748084"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961748084"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961748084"}]},"ts":"1675961748084"} 2023-02-09 16:55:48,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 9fa51a9d7502e93206005ef7befc7fc1, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:48,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=dee274540666147336eaecbeea082487, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961748088"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961748088"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961748088"}]},"ts":"1675961748088"} 2023-02-09 16:55:48,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure dee274540666147336eaecbeea082487, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:48,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9fa51a9d7502e93206005ef7befc7fc1, NAME => 'testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 16:55:48,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:48,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:48,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,241 INFO [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,242 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1 2023-02-09 16:55:48,242 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1 2023-02-09 16:55:48,242 INFO [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9fa51a9d7502e93206005ef7befc7fc1 columnFamilyName cf1 2023-02-09 16:55:48,254 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4-bottom 2023-02-09 16:55:48,257 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622-bottom 2023-02-09 16:55:48,260 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6-bottom 2023-02-09 16:55:48,264 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693-bottom 2023-02-09 16:55:48,267 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0-bottom 2023-02-09 16:55:48,270 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8-bottom 2023-02-09 16:55:48,273 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1-bottom 2023-02-09 16:55:48,273 INFO [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(310): Store=9fa51a9d7502e93206005ef7befc7fc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:48,273 INFO [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,274 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2 2023-02-09 16:55:48,274 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2 2023-02-09 16:55:48,275 INFO [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9fa51a9d7502e93206005ef7befc7fc1 columnFamilyName cf2 2023-02-09 16:55:48,285 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709-bottom 2023-02-09 16:55:48,288 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b-bottom 2023-02-09 16:55:48,292 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0-bottom 2023-02-09 16:55:48,295 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002-bottom 2023-02-09 16:55:48,298 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a-bottom 2023-02-09 16:55:48,301 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139-bottom 2023-02-09 16:55:48,305 DEBUG [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3-bottom 2023-02-09 16:55:48,305 INFO [StoreOpener-9fa51a9d7502e93206005ef7befc7fc1-1] regionserver.HStore(310): Store=9fa51a9d7502e93206005ef7befc7fc1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:48,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:48,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:48,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9fa51a9d7502e93206005ef7befc7fc1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64373412, jitterRate=-0.04076141119003296}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:48,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9fa51a9d7502e93206005ef7befc7fc1: Running coprocessor pre-open hook at 1675961748240Writing region info on filesystem at 1675961748240Initializing all the Stores at 1675961748241 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961748241Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961748241Cleaning up temporary data from old regions at 1675961748308 (+67 ms)Cleaning up detritus from prior splits at 1675961748308Running coprocessor post-open hooks at 1675961748311 (+3 ms)Region opened successfully at 1675961748311 2023-02-09 16:55:48,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1., pid=117, masterSystemTime=1675961748237 2023-02-09 16:55:48,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,313 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:48,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,313 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,313 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:48,313 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1912): 9fa51a9d7502e93206005ef7befc7fc1/cf1 is initiating minor compaction (all files) 2023-02-09 16:55:48,313 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,313 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9fa51a9d7502e93206005ef7befc7fc1/cf1 in testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,313 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1912): 9fa51a9d7502e93206005ef7befc7fc1/cf2 is initiating minor compaction (all files) 2023-02-09 16:55:48,313 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9fa51a9d7502e93206005ef7befc7fc1/cf2 in testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,313 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/.tmp, totalSize=34.9 K 2023-02-09 16:55:48,313 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/.tmp, totalSize=34.9 K 2023-02-09 16:55:48,314 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961743904 2023-02-09 16:55:48,314 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961743904 2023-02-09 16:55:48,314 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961744071 2023-02-09 16:55:48,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:48,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dee274540666147336eaecbeea082487, NAME => 'testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 16:55:48,315 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961744071 2023-02-09 16:55:48,315 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961744255 2023-02-09 16:55:48,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=9fa51a9d7502e93206005ef7befc7fc1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:48,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961748315"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961748315"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961748315"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961748315"}]},"ts":"1675961748315"} 2023-02-09 16:55:48,315 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961744255 2023-02-09 16:55:48,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution dee274540666147336eaecbeea082487 2023-02-09 16:55:48,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:48,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for dee274540666147336eaecbeea082487 2023-02-09 16:55:48,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for dee274540666147336eaecbeea082487 2023-02-09 16:55:48,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-09 16:55:48,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 9fa51a9d7502e93206005ef7befc7fc1, server=jenkins-hbase4.apache.org,33383,1675961647775 in 230 msec 2023-02-09 16:55:48,318 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961744443 2023-02-09 16:55:48,318 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961744443 2023-02-09 16:55:48,319 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961744627 2023-02-09 16:55:48,319 INFO [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dee274540666147336eaecbeea082487 2023-02-09 16:55:48,319 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961744627 2023-02-09 16:55:48,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN in 388 msec 2023-02-09 16:55:48,319 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961744804 2023-02-09 16:55:48,320 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961744804 2023-02-09 16:55:48,320 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961744995 2023-02-09 16:55:48,320 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961744995 2023-02-09 16:55:48,320 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1 2023-02-09 16:55:48,321 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1 2023-02-09 16:55:48,321 INFO [StoreOpener-dee274540666147336eaecbeea082487-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dee274540666147336eaecbeea082487 columnFamilyName cf1 2023-02-09 16:55:48,334 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] throttle.PressureAwareThroughputController(145): 9fa51a9d7502e93206005ef7befc7fc1#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:48,334 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9fa51a9d7502e93206005ef7befc7fc1#cf1#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:48,357 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4-top 2023-02-09 16:55:48,360 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/.tmp/cf1/7f4f863964e7493dbd1b5fb09d747469 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/7f4f863964e7493dbd1b5fb09d747469 2023-02-09 16:55:48,360 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/.tmp/cf2/23960a0dbe3343ccbafbface93487593 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/23960a0dbe3343ccbafbface93487593 2023-02-09 16:55:48,361 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622-top 2023-02-09 16:55:48,364 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9fa51a9d7502e93206005ef7befc7fc1/cf1 of 9fa51a9d7502e93206005ef7befc7fc1 into 7f4f863964e7493dbd1b5fb09d747469(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:48,365 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9fa51a9d7502e93206005ef7befc7fc1: 2023-02-09 16:55:48,365 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1., storeName=9fa51a9d7502e93206005ef7befc7fc1/cf1, priority=9, startTime=1675961748312; duration=0sec 2023-02-09 16:55:48,365 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,365 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6-top 2023-02-09 16:55:48,365 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9fa51a9d7502e93206005ef7befc7fc1/cf2 of 9fa51a9d7502e93206005ef7befc7fc1 into 23960a0dbe3343ccbafbface93487593(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:48,366 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9fa51a9d7502e93206005ef7befc7fc1: 2023-02-09 16:55:48,366 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1., storeName=9fa51a9d7502e93206005ef7befc7fc1/cf2, priority=9, startTime=1675961748313; duration=0sec 2023-02-09 16:55:48,366 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,368 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693-top 2023-02-09 16:55:48,371 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0-top 2023-02-09 16:55:48,374 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8-top 2023-02-09 16:55:48,376 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1-top 2023-02-09 16:55:48,377 INFO [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(310): Store=dee274540666147336eaecbeea082487/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:48,377 INFO [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region dee274540666147336eaecbeea082487 2023-02-09 16:55:48,378 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2 2023-02-09 16:55:48,378 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2 2023-02-09 16:55:48,378 INFO [StoreOpener-dee274540666147336eaecbeea082487-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dee274540666147336eaecbeea082487 columnFamilyName cf2 2023-02-09 16:55:48,391 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709-top 2023-02-09 16:55:48,393 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b-top 2023-02-09 16:55:48,396 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0-top 2023-02-09 16:55:48,399 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002-top 2023-02-09 16:55:48,402 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a-top 2023-02-09 16:55:48,405 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139-top 2023-02-09 16:55:48,408 DEBUG [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3-top 2023-02-09 16:55:48,408 INFO [StoreOpener-dee274540666147336eaecbeea082487-1] regionserver.HStore(310): Store=dee274540666147336eaecbeea082487/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:48,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487 2023-02-09 16:55:48,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487 2023-02-09 16:55:48,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:48,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for dee274540666147336eaecbeea082487 2023-02-09 16:55:48,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened dee274540666147336eaecbeea082487; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68701779, jitterRate=0.02373628318309784}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:48,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for dee274540666147336eaecbeea082487: Running coprocessor pre-open hook at 1675961748315Writing region info on filesystem at 1675961748315Initializing all the Stores at 1675961748318 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961748318Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961748319 (+1 ms)Cleaning up temporary data from old regions at 1675961748409 (+90 ms)Cleaning up detritus from prior splits at 1675961748410 (+1 ms)Running coprocessor post-open hooks at 1675961748412 (+2 ms)Region opened successfully at 1675961748412 2023-02-09 16:55:48,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487., pid=118, masterSystemTime=1675961748237 2023-02-09 16:55:48,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:55:48,414 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:48,414 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:48,414 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,414 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1912): dee274540666147336eaecbeea082487/cf1 is initiating minor compaction (all files) 2023-02-09 16:55:48,414 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,414 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1912): dee274540666147336eaecbeea082487/cf2 is initiating minor compaction (all files) 2023-02-09 16:55:48,414 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of dee274540666147336eaecbeea082487/cf1 in testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,414 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2258): Starting compaction of dee274540666147336eaecbeea082487/cf2 in testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,414 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4-top] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/.tmp, totalSize=34.9 K 2023-02-09 16:55:48,414 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a-top] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/.tmp, totalSize=34.9 K 2023-02-09 16:55:48,415 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961743904 2023-02-09 16:55:48,415 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961743904 2023-02-09 16:55:48,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,415 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961744071 2023-02-09 16:55:48,416 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961744071 2023-02-09 16:55:48,416 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=dee274540666147336eaecbeea082487, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,416 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961744255 2023-02-09 16:55:48,416 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961744255 2023-02-09 16:55:48,416 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961748415"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961748415"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961748415"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961748415"}]},"ts":"1675961748415"} 2023-02-09 16:55:48,416 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961744443 2023-02-09 16:55:48,416 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961744443 2023-02-09 16:55:48,416 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675961744627 2023-02-09 16:55:48,417 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675961744627 2023-02-09 16:55:48,417 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675961744804 2023-02-09 16:55:48,417 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] compactions.Compactor(207): Compacting 21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961744995 2023-02-09 16:55:48,417 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675961744804 2023-02-09 16:55:48,418 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] compactions.Compactor(207): Compacting 77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961744995 2023-02-09 16:55:48,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-09 16:55:48,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure dee274540666147336eaecbeea082487, server=jenkins-hbase4.apache.org,33383,1675961647775 in 328 msec 2023-02-09 16:55:48,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-09 16:55:48,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN in 490 msec 2023-02-09 16:55:48,421 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 16:55:48,421 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:48,421 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 16:55:48,428 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] throttle.PressureAwareThroughputController(145): dee274540666147336eaecbeea082487#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:48,428 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] throttle.PressureAwareThroughputController(145): dee274540666147336eaecbeea082487#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:48,430 INFO [Listener at localhost/39015] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,430 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 16:55:48,430 INFO [Listener at localhost/39015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:48,431 INFO [Listener at localhost/39015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:48,431 ERROR [Listener at localhost/39015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 16:55:48,431 INFO [Listener at localhost/39015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 16:55:48,431 INFO [Listener at localhost/39015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 16:55:48,431 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 16:55:48,432 INFO [Listener at localhost/39015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 16:55:48,432 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 16:55:48,433 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-09 16:55:48,433 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 16:55:48,433 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 16:55:48,433 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-09 16:55:48,433 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 16:55:48,434 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9a0852ba4a2981d6960c832753abdda, daughterA=e30459e13786502111687c71f7099b6b, daughterB=5c7a74a2d1d2af537583d0338c22a787 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=17bc3062a59e914ed503fb224f492a7a, daughterA=3886eaed3353b979029f0f97c1c5b489, daughterB=0e0dc67407eec4115a0c4071ea62b937 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 16:55:48,435 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=c307239a1f0f6e7432449ec50648abae, daughterA=5dbef3147b010b12df9eb8c5e71c12c8, daughterB=614b0c169020f96ce7313ce87ffc96ce 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=c307239a1f0f6e7432449ec50648abae, ASSIGN 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2c16f3ae420a962237b43fe9bcb426dd, daughterA=8b180db08753a36ce5add8496d915265, daughterB=ac7cbf4002f2c7ee631b20295b01672c 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 16:55:48,436 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=7e8ba877d307230da0baa92e22cac26a, daughterA=4fa71edecc82d2950cbaa2c51d57f16c, daughterB=c9f56ee2a4d5fc619dd62cf9306271f6 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43347,1675961647698, table=testRollbackForSplitTableRegionWithReplica, region=aa6a554ecfe82c532d26eba502eca4e9 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=82b4da9176b3dc6382be1c3c41771bc2, daughterA=fae09ffd80364e99c8280e79c338a46e, daughterB=bb479697f216cd634aa58291e084f4dc 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:48,437 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 16:55:48,438 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:48,438 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:48,438 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 16:55:48,438 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:48,439 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,439 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN 2023-02-09 16:55:48,439 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,439 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN 2023-02-09 16:55:48,439 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN 2023-02-09 16:55:48,439 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 9fa51a9d7502e93206005ef7befc7fc1, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure dee274540666147336eaecbeea082487, server=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 held the lock before restarting, call acquireLock to restore it. 2023-02-09 16:55:48,440 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,440 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,440 INFO [Listener at localhost/39015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 8dd6d0e199e1c44e37d692efb64f3fa5, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 9fa51a9d7502e93206005ef7befc7fc1, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:48,440 DEBUG [Listener at localhost/39015] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure dee274540666147336eaecbeea082487, server=jenkins-hbase4.apache.org,33383,1675961647775 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 16:55:48,441 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 16:55:48,441 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 16:55:48,442 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698, table=hbase:meta, region=1588230740 2023-02-09 16:55:48,442 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675961778442 2023-02-09 16:55:48,442 DEBUG [Listener at localhost/39015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 16:55:48,444 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 2023-02-09 16:55:48,446 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=2f94cc9ee2fe56d77bbf16d8818a2c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:48,447 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=8dd6d0e199e1c44e37d692efb64f3fa5, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=2 2023-02-09 16:55:48,447 INFO [Listener at localhost/39015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33383,1675961647775, table=testRecoveryAndDoubleExecution, region=8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,447 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=9fa51a9d7502e93206005ef7befc7fc1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=86 2023-02-09 16:55:48,447 DEBUG [Listener at localhost/39015] assignment.RegionStateStore(147): Load hbase:meta entry region=dee274540666147336eaecbeea082487, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33383,1675961647775, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775, openSeqNum=86 2023-02-09 16:55:48,447 INFO [Listener at localhost/39015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 16:55:48,447 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675961808447 2023-02-09 16:55:48,447 INFO [Listener at localhost/39015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675961868447 2023-02-09 16:55:48,447 INFO [Listener at localhost/39015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 16:55:48,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 16:55:48,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=8dd6d0e199e1c44e37d692efb64f3fa5, daughterA=9fa51a9d7502e93206005ef7befc7fc1, daughterB=dee274540666147336eaecbeea082487 in 3.1930 sec 2023-02-09 16:55:48,452 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/.tmp/cf1/292cccc5d00e4533b000af23672acff1 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/292cccc5d00e4533b000af23672acff1 2023-02-09 16:55:48,453 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/.tmp/cf2/5d72266a72d740a7919bc13281897243 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/5d72266a72d740a7919bc13281897243 2023-02-09 16:55:48,457 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dee274540666147336eaecbeea082487/cf1 of dee274540666147336eaecbeea082487 into 292cccc5d00e4533b000af23672acff1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:48,457 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for dee274540666147336eaecbeea082487: 2023-02-09 16:55:48,457 INFO [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487., storeName=dee274540666147336eaecbeea082487/cf1, priority=9, startTime=1675961748413; duration=0sec 2023-02-09 16:55:48,457 DEBUG [RS:2;jenkins-hbase4:33383-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,457 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in dee274540666147336eaecbeea082487/cf2 of dee274540666147336eaecbeea082487 into 5d72266a72d740a7919bc13281897243(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:48,457 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for dee274540666147336eaecbeea082487: 2023-02-09 16:55:48,457 INFO [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487., storeName=dee274540666147336eaecbeea082487/cf2, priority=9, startTime=1675961748414; duration=0sec 2023-02-09 16:55:48,457 DEBUG [RS:2;jenkins-hbase4:33383-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:48,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33383] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:56108 deadline: 1675961808698, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5. is not online on jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,811 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:55:48,816 INFO [Listener at localhost/39015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 16:55:48,817 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:55:48,825 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:48,825 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:48,826 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:48,827 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:48,827 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:48,827 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-09 16:55:48,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-09 16:55:48,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:48,830 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:48,830 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:48,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 16:55:48,830 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961748830"}]},"ts":"1675961748830"} 2023-02-09 16:55:48,831 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 16:55:48,833 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-09 16:55:48,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, UNASSIGN}] 2023-02-09 16:55:48,834 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, UNASSIGN 2023-02-09 16:55:48,835 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=dee274540666147336eaecbeea082487, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,835 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961748835"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961748835"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961748835"}]},"ts":"1675961748835"} 2023-02-09 16:55:48,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure dee274540666147336eaecbeea082487, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:48,837 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, UNASSIGN 2023-02-09 16:55:48,838 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=9fa51a9d7502e93206005ef7befc7fc1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:48,838 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961748838"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961748838"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961748838"}]},"ts":"1675961748838"} 2023-02-09 16:55:48,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 9fa51a9d7502e93206005ef7befc7fc1, server=jenkins-hbase4.apache.org,33383,1675961647775}] 2023-02-09 16:55:48,931 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:48,931 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:48,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 16:55:48,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dee274540666147336eaecbeea082487 2023-02-09 16:55:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing dee274540666147336eaecbeea082487, disabling compactions & flushes 2023-02-09 16:55:48,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. after waiting 0 ms 2023-02-09 16:55:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:48,988 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4-top] to archive 2023-02-09 16:55:48,989 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:48,991 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,993 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,994 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,995 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,996 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,997 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:48,999 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,000 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a-top] to archive 2023-02-09 16:55:49,001 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:49,002 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,004 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,005 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,006 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,007 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,008 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,009 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:55:49,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:49,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487. 2023-02-09 16:55:49,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for dee274540666147336eaecbeea082487: Waiting for close lock at 1675961748988Running coprocessor pre-close hooks at 1675961748988Disabling compacts and flushes for region at 1675961748988Disabling writes for close at 1675961748988Writing region close event to WAL at 1675961749010 (+22 ms)Running coprocessor post-close hooks at 1675961749014 (+4 ms)Closed at 1675961749014 2023-02-09 16:55:49,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dee274540666147336eaecbeea082487 2023-02-09 16:55:49,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:49,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9fa51a9d7502e93206005ef7befc7fc1, disabling compactions & flushes 2023-02-09 16:55:49,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:49,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:49,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. after waiting 0 ms 2023-02-09 16:55:49,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:49,017 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=dee274540666147336eaecbeea082487, regionState=CLOSED 2023-02-09 16:55:49,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961749017"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961749017"}]},"ts":"1675961749017"} 2023-02-09 16:55:49,018 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4-bottom] to archive 2023-02-09 16:55:49,019 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:49,021 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-09 16:55:49,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure dee274540666147336eaecbeea082487, server=jenkins-hbase4.apache.org,33383,1675961647775 in 183 msec 2023-02-09 16:55:49,022 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=dee274540666147336eaecbeea082487, UNASSIGN in 187 msec 2023-02-09 16:55:49,023 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,024 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,025 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,026 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,026 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,028 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a-bottom] to archive 2023-02-09 16:55:49,029 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:49,030 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,031 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,032 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,033 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,034 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,035 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,036 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:55:49,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:49,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1. 2023-02-09 16:55:49,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9fa51a9d7502e93206005ef7befc7fc1: Waiting for close lock at 1675961749017Running coprocessor pre-close hooks at 1675961749017Disabling compacts and flushes for region at 1675961749017Disabling writes for close at 1675961749017Writing region close event to WAL at 1675961749036 (+19 ms)Running coprocessor post-close hooks at 1675961749039 (+3 ms)Closed at 1675961749039 2023-02-09 16:55:49,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:49,042 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=9fa51a9d7502e93206005ef7befc7fc1, regionState=CLOSED 2023-02-09 16:55:49,042 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675961749042"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961749042"}]},"ts":"1675961749042"} 2023-02-09 16:55:49,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-09 16:55:49,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 9fa51a9d7502e93206005ef7befc7fc1, server=jenkins-hbase4.apache.org,33383,1675961647775 in 204 msec 2023-02-09 16:55:49,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-09 16:55:49,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=9fa51a9d7502e93206005ef7befc7fc1, UNASSIGN in 210 msec 2023-02-09 16:55:49,046 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961749046"}]},"ts":"1675961749046"} 2023-02-09 16:55:49,047 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 16:55:49,048 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-09 16:55:49,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 220 msec 2023-02-09 16:55:49,132 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:49,132 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:49,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 16:55:49,133 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-09 16:55:49,133 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:49,134 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:49,134 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-09 16:55:49,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:49,136 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:49,137 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:49,137 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:49,137 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 16:55:49,142 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,142 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:49,142 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487 2023-02-09 16:55:49,144 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/recovered.edits] 2023-02-09 16:55:49,145 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/recovered.edits] 2023-02-09 16:55:49,145 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/recovered.edits] 2023-02-09 16:55:49,148 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/.regioninfo to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/.regioninfo 2023-02-09 16:55:49,148 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/292cccc5d00e4533b000af23672acff1 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf1/292cccc5d00e4533b000af23672acff1 2023-02-09 16:55:49,149 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/21c2bd5f19714e4d8cee65684f541aa4 2023-02-09 16:55:49,150 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/2411c4999cbf48608455a738a11a2622 2023-02-09 16:55:49,151 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/5d72266a72d740a7919bc13281897243 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/cf2/5d72266a72d740a7919bc13281897243 2023-02-09 16:55:49,151 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,151 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/58010a1dedb54abca0958a6d858c75c6 2023-02-09 16:55:49,153 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,153 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/88fffc7261934215858fe2d39d84d693 2023-02-09 16:55:49,154 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/.regioninfo to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/.regioninfo 2023-02-09 16:55:49,154 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,154 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/a331a6db0abf4663877c1cbd095967e0 2023-02-09 16:55:49,155 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,155 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/bc9546020eb14893a2abac5d2609a3d8 2023-02-09 16:55:49,156 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,157 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf1/d67176a92656446c90e1c07119e978b1 2023-02-09 16:55:49,159 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,161 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,161 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/21c2bd5f19714e4d8cee65684f541aa4.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,161 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/1130ee0cefd849bf8b70070b44127709 2023-02-09 16:55:49,162 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/2411c4999cbf48608455a738a11a2622.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,163 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/227c5f0fc02d456dbec680182669da5b 2023-02-09 16:55:49,164 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/58010a1dedb54abca0958a6d858c75c6.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,164 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/56164a8006714149a4b1bbf6385e72d0 2023-02-09 16:55:49,164 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,166 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/6825808da9a54b4fb43d0d81e5c4b002 2023-02-09 16:55:49,166 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/88fffc7261934215858fe2d39d84d693.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,166 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,174 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/77d08aca744c4a2e9f9443da7b73891a 2023-02-09 16:55:49,174 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/a331a6db0abf4663877c1cbd095967e0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,175 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,175 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/ca993c9199d14e1faf1b00c089b16139 2023-02-09 16:55:49,176 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/bc9546020eb14893a2abac5d2609a3d8.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,176 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,177 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf1/d67176a92656446c90e1c07119e978b1.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,177 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/cf2/cc64d87b6ee846f3b42dffc52306c6a3 2023-02-09 16:55:49,178 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,180 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,180 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/recovered.edits/85.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5/recovered.edits/85.seqid 2023-02-09 16:55:49,181 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,181 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/1130ee0cefd849bf8b70070b44127709.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,181 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/9fa51a9d7502e93206005ef7befc7fc1/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,182 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/227c5f0fc02d456dbec680182669da5b.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,184 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/56164a8006714149a4b1bbf6385e72d0.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,184 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/7f4f863964e7493dbd1b5fb09d747469 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf1/7f4f863964e7493dbd1b5fb09d747469 2023-02-09 16:55:49,185 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/6825808da9a54b4fb43d0d81e5c4b002.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,186 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/77d08aca744c4a2e9f9443da7b73891a.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,186 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/23960a0dbe3343ccbafbface93487593 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/cf2/23960a0dbe3343ccbafbface93487593 2023-02-09 16:55:49,187 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/ca993c9199d14e1faf1b00c089b16139.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,189 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/dee274540666147336eaecbeea082487/cf2/cc64d87b6ee846f3b42dffc52306c6a3.8dd6d0e199e1c44e37d692efb64f3fa5 2023-02-09 16:55:49,189 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1/recovered.edits/90.seqid 2023-02-09 16:55:49,189 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/9fa51a9d7502e93206005ef7befc7fc1 2023-02-09 16:55:49,191 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487/recovered.edits/90.seqid 2023-02-09 16:55:49,191 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testRecoveryAndDoubleExecution/dee274540666147336eaecbeea082487 2023-02-09 16:55:49,191 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 16:55:49,194 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:49,195 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-09 16:55:49,197 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-09 16:55:49,198 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:49,198 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-09 16:55:49,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961749198"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:49,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961749198"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:49,198 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961749198"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:49,199 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:55:49,199 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8dd6d0e199e1c44e37d692efb64f3fa5, NAME => 'testRecoveryAndDoubleExecution,,1675961743401.8dd6d0e199e1c44e37d692efb64f3fa5.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 9fa51a9d7502e93206005ef7befc7fc1, NAME => 'testRecoveryAndDoubleExecution,,1675961745254.9fa51a9d7502e93206005ef7befc7fc1.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => dee274540666147336eaecbeea082487, NAME => 'testRecoveryAndDoubleExecution,41,1675961745254.dee274540666147336eaecbeea082487.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 16:55:49,199 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-09 16:55:49,199 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961749199"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:49,200 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-09 16:55:49,202 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 16:55:49,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 68 msec 2023-02-09 16:55:49,238 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:49,238 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:49,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 16:55:49,239 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-09 16:55:49,254 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 367) - Thread LEAK? -, OpenFileDescriptor=825 (was 742) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 103), ProcessCount=168 (was 168), AvailableMemoryMB=2901 (was 2949) 2023-02-09 16:55:49,267 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=825, MaxFileDescriptor=60000, SystemLoadAverage=95, ProcessCount=168, AvailableMemoryMB=2901 2023-02-09 16:55:49,267 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:49,267 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:49,267 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:49,269 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:55:49,269 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:55:49,270 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,272 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:55:49,272 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:55:49,273 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,274 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 empty. 2023-02-09 16:55:49,274 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,274 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 16:55:49,283 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 16:55:49,284 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => e223b141ea1d973993597987c0d85170, NAME => 'testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:55:49,293 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:49,293 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing e223b141ea1d973993597987c0d85170, disabling compactions & flushes 2023-02-09 16:55:49,293 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,293 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,293 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. after waiting 0 ms 2023-02-09 16:55:49,293 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,293 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,293 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for e223b141ea1d973993597987c0d85170: Waiting for close lock at 1675961749293Disabling compacts and flushes for region at 1675961749293Disabling writes for close at 1675961749293Writing region close event to WAL at 1675961749293Closed at 1675961749293 2023-02-09 16:55:49,295 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:55:49,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961749296"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961749296"}]},"ts":"1675961749296"} 2023-02-09 16:55:49,297 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:55:49,298 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:55:49,298 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961749298"}]},"ts":"1675961749298"} 2023-02-09 16:55:49,299 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-09 16:55:49,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:55:49,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:55:49,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:55:49,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:55:49,303 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:55:49,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, ASSIGN}] 2023-02-09 16:55:49,305 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, ASSIGN 2023-02-09 16:55:49,305 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:55:49,456 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:55:49,457 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=e223b141ea1d973993597987c0d85170, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:49,457 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961749457"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961749457"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961749457"}]},"ts":"1675961749457"} 2023-02-09 16:55:49,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure e223b141ea1d973993597987c0d85170, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:49,578 ERROR [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid 762ffd31-8114-414e-bb4b-20a4d577f100) service to localhost/127.0.0.1:44699 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:55:49,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e223b141ea1d973993597987c0d85170, NAME => 'testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:55:49,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:49,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:49,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,616 INFO [StoreOpener-e223b141ea1d973993597987c0d85170-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,617 DEBUG [StoreOpener-e223b141ea1d973993597987c0d85170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1 2023-02-09 16:55:49,618 DEBUG [StoreOpener-e223b141ea1d973993597987c0d85170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1 2023-02-09 16:55:49,618 INFO [StoreOpener-e223b141ea1d973993597987c0d85170-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e223b141ea1d973993597987c0d85170 columnFamilyName cf1 2023-02-09 16:55:49,618 INFO [StoreOpener-e223b141ea1d973993597987c0d85170-1] regionserver.HStore(310): Store=e223b141ea1d973993597987c0d85170/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:49,618 INFO [StoreOpener-e223b141ea1d973993597987c0d85170-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,619 DEBUG [StoreOpener-e223b141ea1d973993597987c0d85170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2 2023-02-09 16:55:49,620 DEBUG [StoreOpener-e223b141ea1d973993597987c0d85170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2 2023-02-09 16:55:49,620 INFO [StoreOpener-e223b141ea1d973993597987c0d85170-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e223b141ea1d973993597987c0d85170 columnFamilyName cf2 2023-02-09 16:55:49,620 INFO [StoreOpener-e223b141ea1d973993597987c0d85170-1] regionserver.HStore(310): Store=e223b141ea1d973993597987c0d85170/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:49,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:49,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e223b141ea1d973993597987c0d85170 2023-02-09 16:55:49,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:55:49,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e223b141ea1d973993597987c0d85170; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68840183, jitterRate=0.02579866349697113}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:49,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e223b141ea1d973993597987c0d85170: Running coprocessor pre-open hook at 1675961749615Writing region info on filesystem at 1675961749615Initializing all the Stores at 1675961749616 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961749616Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961749616Cleaning up temporary data from old regions at 1675961749622 (+6 ms)Cleaning up detritus from prior splits at 1675961749622Running coprocessor post-open hooks at 1675961749626 (+4 ms)Region opened successfully at 1675961749626 2023-02-09 16:55:49,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170., pid=127, masterSystemTime=1675961749610 2023-02-09 16:55:49,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,628 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=e223b141ea1d973993597987c0d85170, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:49,628 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961749628"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961749628"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961749628"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961749628"}]},"ts":"1675961749628"} 2023-02-09 16:55:49,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-09 16:55:49,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure e223b141ea1d973993597987c0d85170, server=jenkins-hbase4.apache.org,43347,1675961647698 in 171 msec 2023-02-09 16:55:49,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-09 16:55:49,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, ASSIGN in 327 msec 2023-02-09 16:55:49,632 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:55:49,633 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961749632"}]},"ts":"1675961749632"} 2023-02-09 16:55:49,633 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-09 16:55:49,635 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:55:49,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 365 msec 2023-02-09 16:55:49,777 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:49,777 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:49,778 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:49,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,779 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:49,779 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:49,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 16:55:49,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 16:55:49,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,780 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,781 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:49,781 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:49,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:49,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:49,781 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:49,781 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:49,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:49,781 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:49,782 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:49,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:49,783 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:49,783 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. started... 2023-02-09 16:55:49,784 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:55:49,785 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,785 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,785 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,785 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:49,786 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:49,786 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:49,793 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/1bd1d7eef8be43439117973506434bb6 2023-02-09 16:55:50,203 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/772053991fe540da82b38afb1593c49a 2023-02-09 16:55:50,207 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/1bd1d7eef8be43439117973506434bb6 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6 2023-02-09 16:55:50,211 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:50,211 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/772053991fe540da82b38afb1593c49a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a 2023-02-09 16:55:50,214 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:50,215 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 431ms, sequenceid=6, compaction requested=false 2023-02-09 16:55:50,215 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,216 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e223b141ea1d973993597987c0d85170: 2023-02-09 16:55:50,216 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,216 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,218 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,218 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,219 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,219 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 16:55:50,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e09f83c[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:50,220 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 16:55:50,220 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,221 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,221 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,221 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,222 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,222 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,222 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:50,223 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,223 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,223 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,224 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,224 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:50,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:55:50,226 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 16:55:50,227 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:50,227 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:50,227 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,227 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:50,229 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,229 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,229 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,230 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,231 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,232 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,232 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,233 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,238 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,238 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,238 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,238 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,238 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,238 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,239 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:50,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:50,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,239 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,240 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,240 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,240 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,240 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,240 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,240 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,340 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:50,340 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:50,340 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:50,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:50,350 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:50,351 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:50,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:50,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:50,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,353 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 16:55:50,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 16:55:50,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,353 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,354 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,354 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,354 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,354 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,355 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,355 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,357 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,357 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,357 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,357 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,357 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:50,358 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. started... 2023-02-09 16:55:50,358 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,360 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,360 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,360 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,368 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/a646ac64bc974db78d4209dd62a63e6a 2023-02-09 16:55:50,379 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/551378437769498e8a633a8d6a54a7b7 2023-02-09 16:55:50,382 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/a646ac64bc974db78d4209dd62a63e6a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a 2023-02-09 16:55:50,386 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:50,387 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/551378437769498e8a633a8d6a54a7b7 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7 2023-02-09 16:55:50,390 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:50,390 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 32ms, sequenceid=19, compaction requested=false 2023-02-09 16:55:50,391 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e223b141ea1d973993597987c0d85170: 2023-02-09 16:55:50,391 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,393 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,393 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,393 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,393 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 16:55:50,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bc969eb[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:50,395 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 16:55:50,395 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,396 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:50,397 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:50,396 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,397 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,398 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,398 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,399 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,399 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,399 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:50,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:55:50,401 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 16:55:50,401 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:50,401 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:50,401 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,401 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:50,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,403 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,404 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,404 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,404 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,405 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,406 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,406 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,407 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,409 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,410 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:50,409 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,409 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:50,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,410 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,411 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,411 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,510 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:50,511 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:50,511 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:50,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:50,521 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:50,521 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:50,522 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:50,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:50,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,523 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,523 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,523 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 16:55:50,523 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 16:55:50,524 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,524 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,525 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,525 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,525 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,525 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,526 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,526 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,527 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,527 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,527 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. started... 2023-02-09 16:55:50,527 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:50,529 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,542 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/4cccf561ba9b4cbebadbb372aed8e186 2023-02-09 16:55:50,554 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/e1cd007bcfdb4160b8a06c6ca1454158 2023-02-09 16:55:50,557 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/4cccf561ba9b4cbebadbb372aed8e186 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186 2023-02-09 16:55:50,560 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:50,561 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/e1cd007bcfdb4160b8a06c6ca1454158 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158 2023-02-09 16:55:50,564 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:50,564 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 37ms, sequenceid=32, compaction requested=true 2023-02-09 16:55:50,564 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e223b141ea1d973993597987c0d85170: 2023-02-09 16:55:50,565 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,565 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,566 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,566 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,566 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,566 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 16:55:50,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d56a9f1[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:50,568 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 16:55:50,568 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,569 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:50,570 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:50,574 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,575 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,575 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,575 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,575 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,576 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:50,576 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,576 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:50,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:50,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:55:50,578 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 16:55:50,578 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:50,578 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:50,578 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,578 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,580 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:50,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,580 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,581 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,581 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:50,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,582 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,583 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:50,583 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,585 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,586 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,586 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:50,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,586 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:50,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,586 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:50,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,587 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,687 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:50,687 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:50,687 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:50,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:50,697 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:50,697 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:50,698 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:50,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:50,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,699 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:50,699 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,699 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 16:55:50,699 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 16:55:50,700 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,700 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:50,702 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,702 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:50,702 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,703 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:50,704 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:50,704 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:50,704 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. started... 2023-02-09 16:55:50,704 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:50,705 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:50,706 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:50,712 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/564978c7573b4208aafddc5f81717507 2023-02-09 16:55:51,122 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/88b52ab9c43c4bcbbeaa4170927016be 2023-02-09 16:55:51,126 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/564978c7573b4208aafddc5f81717507 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507 2023-02-09 16:55:51,129 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:51,129 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/88b52ab9c43c4bcbbeaa4170927016be as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be 2023-02-09 16:55:51,132 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:51,133 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 429ms, sequenceid=45, compaction requested=true 2023-02-09 16:55:51,133 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e223b141ea1d973993597987c0d85170: 2023-02-09 16:55:51,133 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:51,133 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:51,133 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:51,133 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,133 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,134 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,135 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,135 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,135 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,135 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 16:55:51,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5091209d[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:51,137 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 16:55:51,137 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,138 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,138 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,138 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:51,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,139 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,139 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:51,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:51,139 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:51,141 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,141 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,141 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,141 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,142 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,142 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,142 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,142 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,142 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,142 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:51,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:55:51,144 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 16:55:51,144 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:51,144 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:51,144 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,144 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:51,146 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,146 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,146 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,147 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,147 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,147 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,148 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,149 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,149 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,150 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,155 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:51,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:51,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:51,155 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,156 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,256 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:51,256 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:51,256 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:51,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:51,265 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:51,266 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:51,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:51,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:51,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,267 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:51,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 16:55:51,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 16:55:51,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,268 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,269 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,269 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,269 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,269 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,269 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,270 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:51,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:51,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:51,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:51,271 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,271 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. started... 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,271 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,271 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:51,272 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,274 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,274 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,274 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,274 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,274 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,279 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/792d518eafda4b999bcb0cbb1cc54531 2023-02-09 16:55:51,689 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/61aecb055b4b40cdadb0d60bd1f6daab 2023-02-09 16:55:51,693 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/792d518eafda4b999bcb0cbb1cc54531 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531 2023-02-09 16:55:51,696 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:51,696 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/61aecb055b4b40cdadb0d60bd1f6daab as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab 2023-02-09 16:55:51,700 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:51,700 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 429ms, sequenceid=58, compaction requested=true 2023-02-09 16:55:51,701 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e223b141ea1d973993597987c0d85170: 2023-02-09 16:55:51,701 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:51,701 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:51,701 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:51,701 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,701 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,701 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,702 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,702 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,702 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,703 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 16:55:51,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71f8eaab[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:51,704 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 16:55:51,704 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,706 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,706 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:51,705 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:51,705 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:51,707 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,707 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,707 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,709 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,709 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,709 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,709 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,709 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,709 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,709 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:51,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:55:51,712 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 16:55:51,712 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:51,712 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:51,712 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,712 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:51,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,714 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,713 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,714 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,714 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,714 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,715 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,716 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,716 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,717 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,719 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,719 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:51,719 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:51,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,719 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,720 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,763 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:55:51,820 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:51,820 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:51,820 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:51,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:51,830 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:51,831 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:51,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:51,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:51,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,832 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:51,832 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,833 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 16:55:51,833 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 16:55:51,833 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,833 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,834 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,834 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,834 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,834 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,834 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,834 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:51,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:51,835 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:51,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,835 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:51,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:51,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:51,836 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:51,836 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. started... 2023-02-09 16:55:51,836 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:51,836 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:51,838 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,839 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,839 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,839 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,839 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,839 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,844 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/8cb74e31110245a3a7cc29fb2042def6 2023-02-09 16:55:51,853 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/fb4a56f7896e4ff295dea9f7a3671e9e 2023-02-09 16:55:51,856 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/8cb74e31110245a3a7cc29fb2042def6 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6 2023-02-09 16:55:51,859 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:51,860 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/fb4a56f7896e4ff295dea9f7a3671e9e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e 2023-02-09 16:55:51,863 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:51,864 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 28ms, sequenceid=71, compaction requested=true 2023-02-09 16:55:51,864 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e223b141ea1d973993597987c0d85170: 2023-02-09 16:55:51,864 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:51,864 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:51,864 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:51,864 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,864 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 16:55:51,864 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 16:55:51,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,865 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,865 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,865 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:51,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 16:55:51,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21ae534e[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:51,867 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 16:55:51,867 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,871 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:51,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,871 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,872 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,872 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,872 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:51,872 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:51,871 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 16:55:51,872 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 16:55:51,872 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:51,872 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,873 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,873 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,873 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,873 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,873 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,875 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,875 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 16:55:51,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,875 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,875 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 16:55:51,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:51,878 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 16:55:51,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:51,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:51,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 16:55:51,878 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,878 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:51,879 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,879 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:51,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:51,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,879 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,879 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,879 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:51,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,880 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,880 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,880 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:51,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,881 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:51,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,882 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,882 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,883 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 16:55:51,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:51,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:51,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:51,885 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:51,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:51,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:51,885 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,886 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 16:55:51,986 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:51,986 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:51,986 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:51,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:51,993 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,994 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e223b141ea1d973993597987c0d85170, daughterA=9b00796ea92440e745d6d3ac7e7cce63, daughterB=b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:51,995 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e223b141ea1d973993597987c0d85170, daughterA=9b00796ea92440e745d6d3ac7e7cce63, daughterB=b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:51,995 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e223b141ea1d973993597987c0d85170, daughterA=9b00796ea92440e745d6d3ac7e7cce63, daughterB=b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:51,995 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e223b141ea1d973993597987c0d85170, daughterA=9b00796ea92440e745d6d3ac7e7cce63, daughterB=b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:51,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, UNASSIGN}] 2023-02-09 16:55:51,998 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, UNASSIGN 2023-02-09 16:55:51,998 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=e223b141ea1d973993597987c0d85170, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:51,998 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961751998"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961751998"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961751998"}]},"ts":"1675961751998"} 2023-02-09 16:55:51,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure e223b141ea1d973993597987c0d85170, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:52,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e223b141ea1d973993597987c0d85170, disabling compactions & flushes 2023-02-09 16:55:52,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:52,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:52,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. after waiting 0 ms 2023-02-09 16:55:52,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:52,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e223b141ea1d973993597987c0d85170 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:55:52,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/1a8d884b02134330b8c304bc34c23813 2023-02-09 16:55:52,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/98952011c2ad4b37b3aa7bef176ae0c6 2023-02-09 16:55:52,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf1/1a8d884b02134330b8c304bc34c23813 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813 2023-02-09 16:55:52,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:52,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.tmp/cf2/98952011c2ad4b37b3aa7bef176ae0c6 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6 2023-02-09 16:55:52,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:52,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 38ms, sequenceid=82, compaction requested=true 2023-02-09 16:55:52,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:55:52,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:52,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. 2023-02-09 16:55:52,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e223b141ea1d973993597987c0d85170: Waiting for close lock at 1675961752151Running coprocessor pre-close hooks at 1675961752151Disabling compacts and flushes for region at 1675961752151Disabling writes for close at 1675961752151Obtaining lock to block concurrent updates at 1675961752151Preparing flush snapshotting stores in e223b141ea1d973993597987c0d85170 at 1675961752151Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675961752151Flushing stores of testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. at 1675961752152 (+1 ms)Flushing e223b141ea1d973993597987c0d85170/cf1: creating writer at 1675961752154 (+2 ms)Flushing e223b141ea1d973993597987c0d85170/cf1: appending metadata at 1675961752156 (+2 ms)Flushing e223b141ea1d973993597987c0d85170/cf1: closing flushed file at 1675961752156Flushing e223b141ea1d973993597987c0d85170/cf2: creating writer at 1675961752169 (+13 ms)Flushing e223b141ea1d973993597987c0d85170/cf2: appending metadata at 1675961752171 (+2 ms)Flushing e223b141ea1d973993597987c0d85170/cf2: closing flushed file at 1675961752171Flushing e223b141ea1d973993597987c0d85170/cf1: reopening flushed file at 1675961752181 (+10 ms)Flushing e223b141ea1d973993597987c0d85170/cf2: reopening flushed file at 1675961752185 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e223b141ea1d973993597987c0d85170 in 38ms, sequenceid=82, compaction requested=true at 1675961752189 (+4 ms)Writing region close event to WAL at 1675961752190 (+1 ms)Running coprocessor post-close hooks at 1675961752193 (+3 ms)Closed at 1675961752194 (+1 ms) 2023-02-09 16:55:52,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,196 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=e223b141ea1d973993597987c0d85170, regionState=CLOSED 2023-02-09 16:55:52,196 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961752196"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961752196"}]},"ts":"1675961752196"} 2023-02-09 16:55:52,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-09 16:55:52,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure e223b141ea1d973993597987c0d85170, server=jenkins-hbase4.apache.org,43347,1675961647698 in 198 msec 2023-02-09 16:55:52,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-09 16:55:52,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e223b141ea1d973993597987c0d85170, UNASSIGN in 201 msec 2023-02-09 16:55:52,209 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=e223b141ea1d973993597987c0d85170, threads=14 2023-02-09 16:55:52,209 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,210 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,211 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,211 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,211 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,211 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,216 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be, top=false 2023-02-09 16:55:52,217 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e, top=false 2023-02-09 16:55:52,221 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186, top=false 2023-02-09 16:55:52,221 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab, top=false 2023-02-09 16:55:52,226 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813, top=false 2023-02-09 16:55:52,226 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158, top=false 2023-02-09 16:55:52,227 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6, top=false 2023-02-09 16:55:52,228 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7, top=false 2023-02-09 16:55:52,229 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507, top=false 2023-02-09 16:55:52,230 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531, top=false 2023-02-09 16:55:52,230 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a, top=false 2023-02-09 16:55:52,233 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a, top=false 2023-02-09 16:55:52,233 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6, top=false 2023-02-09 16:55:52,234 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6, top=false 2023-02-09 16:55:52,252 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,256 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,260 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,261 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,261 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,268 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,268 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,268 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,268 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,268 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,269 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,269 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,270 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,270 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,271 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,272 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,272 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,273 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,274 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/.splits/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507 for child: 9b00796ea92440e745d6d3ac7e7cce63, parent: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,274 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,275 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,275 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,277 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,277 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,278 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,278 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,279 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158 for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,279 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e for region: e223b141ea1d973993597987c0d85170 2023-02-09 16:55:52,279 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region e223b141ea1d973993597987c0d85170 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-09 16:55:52,308 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:55:52,310 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:55:52,311 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675961752311"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675961752311"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675961752311"}]},"ts":"1675961752311"} 2023-02-09 16:55:52,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752311"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961752311"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961752311"}]},"ts":"1675961752311"} 2023-02-09 16:55:52,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752311"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961752311"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961752311"}]},"ts":"1675961752311"} 2023-02-09 16:55:52,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, ASSIGN}] 2023-02-09 16:55:52,315 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, ASSIGN 2023-02-09 16:55:52,316 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:55:52,316 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, ASSIGN 2023-02-09 16:55:52,317 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:55:52,466 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:55:52,467 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=9b00796ea92440e745d6d3ac7e7cce63, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,468 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752467"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961752467"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961752467"}]},"ts":"1675961752467"} 2023-02-09 16:55:52,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 9b00796ea92440e745d6d3ac7e7cce63, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:52,471 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b37c49ed7e31887c6eb48358966d0208, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,471 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752471"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961752471"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961752471"}]},"ts":"1675961752471"} 2023-02-09 16:55:52,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure b37c49ed7e31887c6eb48358966d0208, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:52,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:52,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9b00796ea92440e745d6d3ac7e7cce63, NAME => 'testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.', STARTKEY => '', ENDKEY => '71'} 2023-02-09 16:55:52,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:52,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:52,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,624 INFO [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,625 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1 2023-02-09 16:55:52,625 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1 2023-02-09 16:55:52,626 INFO [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b00796ea92440e745d6d3ac7e7cce63 columnFamilyName cf1 2023-02-09 16:55:52,630 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813 2023-02-09 16:55:52,633 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6 2023-02-09 16:55:52,635 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186 2023-02-09 16:55:52,637 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507 2023-02-09 16:55:52,640 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531 2023-02-09 16:55:52,643 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6 2023-02-09 16:55:52,645 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a 2023-02-09 16:55:52,645 INFO [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(310): Store=9b00796ea92440e745d6d3ac7e7cce63/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:52,645 INFO [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,646 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2 2023-02-09 16:55:52,646 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2 2023-02-09 16:55:52,646 INFO [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b00796ea92440e745d6d3ac7e7cce63 columnFamilyName cf2 2023-02-09 16:55:52,650 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7 2023-02-09 16:55:52,652 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab 2023-02-09 16:55:52,655 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a 2023-02-09 16:55:52,657 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be 2023-02-09 16:55:52,660 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6 2023-02-09 16:55:52,662 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158 2023-02-09 16:55:52,665 DEBUG [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e 2023-02-09 16:55:52,665 INFO [StoreOpener-9b00796ea92440e745d6d3ac7e7cce63-1] regionserver.HStore(310): Store=9b00796ea92440e745d6d3ac7e7cce63/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:52,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:52,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:52,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9b00796ea92440e745d6d3ac7e7cce63; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74377966, jitterRate=0.10831806063652039}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:52,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9b00796ea92440e745d6d3ac7e7cce63: Running coprocessor pre-open hook at 1675961752623Writing region info on filesystem at 1675961752623Initializing all the Stores at 1675961752624 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961752624Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961752624Cleaning up temporary data from old regions at 1675961752667 (+43 ms)Cleaning up detritus from prior splits at 1675961752667Running coprocessor post-open hooks at 1675961752669 (+2 ms)Region opened successfully at 1675961752669 2023-02-09 16:55:52,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63., pid=133, masterSystemTime=1675961752620 2023-02-09 16:55:52,669 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:52,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:55:52,670 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:52,670 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:52,670 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 16:55:52,670 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 16:55:52,670 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): 9b00796ea92440e745d6d3ac7e7cce63/cf1 is initiating minor compaction (all files) 2023-02-09 16:55:52,671 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): 9b00796ea92440e745d6d3ac7e7cce63/cf2 is initiating minor compaction (all files) 2023-02-09 16:55:52,671 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9b00796ea92440e745d6d3ac7e7cce63/cf1 in testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:52,671 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9b00796ea92440e745d6d3ac7e7cce63/cf2 in testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:52,671 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/.tmp, totalSize=34.9 K 2023-02-09 16:55:52,671 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/.tmp, totalSize=34.9 K 2023-02-09 16:55:52,671 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961749773 2023-02-09 16:55:52,671 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961750341 2023-02-09 16:55:52,671 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961749773 2023-02-09 16:55:52,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:52,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:52,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:52,672 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961750512 2023-02-09 16:55:52,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b37c49ed7e31887c6eb48358966d0208, NAME => 'testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.', STARTKEY => '71', ENDKEY => ''} 2023-02-09 16:55:52,672 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961750341 2023-02-09 16:55:52,672 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=9b00796ea92440e745d6d3ac7e7cce63, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:52,672 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961750688 2023-02-09 16:55:52,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,672 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752672"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961752672"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961752672"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961752672"}]},"ts":"1675961752672"} 2023-02-09 16:55:52,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:52,672 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961750512 2023-02-09 16:55:52,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,673 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961751257 2023-02-09 16:55:52,673 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961750688 2023-02-09 16:55:52,673 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961751822 2023-02-09 16:55:52,673 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961751257 2023-02-09 16:55:52,674 INFO [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,675 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961751822 2023-02-09 16:55:52,675 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961751987 2023-02-09 16:55:52,675 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961751987 2023-02-09 16:55:52,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-09 16:55:52,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 9b00796ea92440e745d6d3ac7e7cce63, server=jenkins-hbase4.apache.org,43347,1675961647698 in 205 msec 2023-02-09 16:55:52,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, ASSIGN in 363 msec 2023-02-09 16:55:52,685 DEBUG [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/cf1 2023-02-09 16:55:52,685 DEBUG [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/cf1 2023-02-09 16:55:52,685 INFO [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b37c49ed7e31887c6eb48358966d0208 columnFamilyName cf1 2023-02-09 16:55:52,686 INFO [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] regionserver.HStore(310): Store=b37c49ed7e31887c6eb48358966d0208/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:52,686 INFO [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,688 DEBUG [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/cf2 2023-02-09 16:55:52,689 DEBUG [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/cf2 2023-02-09 16:55:52,689 INFO [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b37c49ed7e31887c6eb48358966d0208 columnFamilyName cf2 2023-02-09 16:55:52,690 INFO [StoreOpener-b37c49ed7e31887c6eb48358966d0208-1] regionserver.HStore(310): Store=b37c49ed7e31887c6eb48358966d0208/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:52,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:52,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:52,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b37c49ed7e31887c6eb48358966d0208; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75275296, jitterRate=0.1216893196105957}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:52,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b37c49ed7e31887c6eb48358966d0208: Running coprocessor pre-open hook at 1675961752673Writing region info on filesystem at 1675961752673Initializing all the Stores at 1675961752673Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961752673Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961752673Cleaning up temporary data from old regions at 1675961752693 (+20 ms)Cleaning up detritus from prior splits at 1675961752694 (+1 ms)Running coprocessor post-open hooks at 1675961752697 (+3 ms)Region opened successfully at 1675961752697 2023-02-09 16:55:52,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208., pid=134, masterSystemTime=1675961752620 2023-02-09 16:55:52,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:52,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:52,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=b37c49ed7e31887c6eb48358966d0208, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752700"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961752700"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961752700"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961752700"}]},"ts":"1675961752700"} 2023-02-09 16:55:52,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-09 16:55:52,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure b37c49ed7e31887c6eb48358966d0208, server=jenkins-hbase4.apache.org,43347,1675961647698 in 229 msec 2023-02-09 16:55:52,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-09 16:55:52,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, ASSIGN in 389 msec 2023-02-09 16:55:52,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e223b141ea1d973993597987c0d85170, daughterA=9b00796ea92440e745d6d3ac7e7cce63, daughterB=b37c49ed7e31887c6eb48358966d0208 in 711 msec 2023-02-09 16:55:52,716 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9b00796ea92440e745d6d3ac7e7cce63#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:52,717 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): 9b00796ea92440e745d6d3ac7e7cce63#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:52,739 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/.tmp/cf2/90e71d787b4b4021be4ba60446091160 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/90e71d787b4b4021be4ba60446091160 2023-02-09 16:55:52,740 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/.tmp/cf1/a3741ae28dc844f78309064a6fa7075e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/a3741ae28dc844f78309064a6fa7075e 2023-02-09 16:55:52,744 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9b00796ea92440e745d6d3ac7e7cce63/cf2 of 9b00796ea92440e745d6d3ac7e7cce63 into 90e71d787b4b4021be4ba60446091160(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:52,744 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9b00796ea92440e745d6d3ac7e7cce63: 2023-02-09 16:55:52,744 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63., storeName=9b00796ea92440e745d6d3ac7e7cce63/cf2, priority=9, startTime=1675961752670; duration=0sec 2023-02-09 16:55:52,745 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9b00796ea92440e745d6d3ac7e7cce63/cf1 of 9b00796ea92440e745d6d3ac7e7cce63 into a3741ae28dc844f78309064a6fa7075e(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:52,745 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:52,745 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9b00796ea92440e745d6d3ac7e7cce63: 2023-02-09 16:55:52,745 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63., storeName=9b00796ea92440e745d6d3ac7e7cce63/cf1, priority=9, startTime=1675961752669; duration=0sec 2023-02-09 16:55:52,746 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:52,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43347] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:36868 deadline: 1675961812746, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170. is not online on jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,856 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-09 16:55:52,860 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:52,860 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:52,860 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:52,861 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:52,861 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:52,861 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-09 16:55:52,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-09 16:55:52,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:52,864 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:52,864 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:52,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 16:55:52,865 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961752865"}]},"ts":"1675961752865"} 2023-02-09 16:55:52,866 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-09 16:55:52,867 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-09 16:55:52,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, UNASSIGN}] 2023-02-09 16:55:52,868 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, UNASSIGN 2023-02-09 16:55:52,869 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b37c49ed7e31887c6eb48358966d0208, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,869 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752869"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961752869"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961752869"}]},"ts":"1675961752869"} 2023-02-09 16:55:52,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure b37c49ed7e31887c6eb48358966d0208, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:52,871 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, UNASSIGN 2023-02-09 16:55:52,872 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=9b00796ea92440e745d6d3ac7e7cce63, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:52,872 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961752872"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961752872"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961752872"}]},"ts":"1675961752872"} 2023-02-09 16:55:52,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 9b00796ea92440e745d6d3ac7e7cce63, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:52,965 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:52,965 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:52,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 16:55:53,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:53,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9b00796ea92440e745d6d3ac7e7cce63, disabling compactions & flushes 2023-02-09 16:55:53,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:53,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:53,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. after waiting 0 ms 2023-02-09 16:55:53,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:53,021 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813] to archive 2023-02-09 16:55:53,022 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:53,023 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1bd1d7eef8be43439117973506434bb6 2023-02-09 16:55:53,024 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-a646ac64bc974db78d4209dd62a63e6a 2023-02-09 16:55:53,025 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-4cccf561ba9b4cbebadbb372aed8e186 2023-02-09 16:55:53,026 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-564978c7573b4208aafddc5f81717507 2023-02-09 16:55:53,027 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-792d518eafda4b999bcb0cbb1cc54531 2023-02-09 16:55:53,028 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-8cb74e31110245a3a7cc29fb2042def6 2023-02-09 16:55:53,029 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-1a8d884b02134330b8c304bc34c23813 2023-02-09 16:55:53,030 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6] to archive 2023-02-09 16:55:53,031 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:53,032 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-772053991fe540da82b38afb1593c49a 2023-02-09 16:55:53,033 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-551378437769498e8a633a8d6a54a7b7 2023-02-09 16:55:53,033 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-e1cd007bcfdb4160b8a06c6ca1454158 2023-02-09 16:55:53,034 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-88b52ab9c43c4bcbbeaa4170927016be 2023-02-09 16:55:53,035 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-61aecb055b4b40cdadb0d60bd1f6daab 2023-02-09 16:55:53,035 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-fb4a56f7896e4ff295dea9f7a3671e9e 2023-02-09 16:55:53,036 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/testSplitTableRegionEmptyDaughter=e223b141ea1d973993597987c0d85170-98952011c2ad4b37b3aa7bef176ae0c6 2023-02-09 16:55:53,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:55:53,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:53,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63. 2023-02-09 16:55:53,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9b00796ea92440e745d6d3ac7e7cce63: Waiting for close lock at 1675961753020Running coprocessor pre-close hooks at 1675961753020Disabling compacts and flushes for region at 1675961753020Disabling writes for close at 1675961753021 (+1 ms)Writing region close event to WAL at 1675961753037 (+16 ms)Running coprocessor post-close hooks at 1675961753039 (+2 ms)Closed at 1675961753040 (+1 ms) 2023-02-09 16:55:53,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:53,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:53,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b37c49ed7e31887c6eb48358966d0208, disabling compactions & flushes 2023-02-09 16:55:53,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:53,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:53,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. after waiting 0 ms 2023-02-09 16:55:53,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:53,042 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=9b00796ea92440e745d6d3ac7e7cce63, regionState=CLOSED 2023-02-09 16:55:53,042 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961753042"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961753042"}]},"ts":"1675961753042"} 2023-02-09 16:55:53,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-09 16:55:53,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 9b00796ea92440e745d6d3ac7e7cce63, server=jenkins-hbase4.apache.org,43347,1675961647698 in 170 msec 2023-02-09 16:55:53,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-09 16:55:53,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:53,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208. 2023-02-09 16:55:53,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b37c49ed7e31887c6eb48358966d0208: Waiting for close lock at 1675961753041Running coprocessor pre-close hooks at 1675961753041Disabling compacts and flushes for region at 1675961753041Disabling writes for close at 1675961753042 (+1 ms)Writing region close event to WAL at 1675961753042Running coprocessor post-close hooks at 1675961753045 (+3 ms)Closed at 1675961753045 2023-02-09 16:55:53,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b00796ea92440e745d6d3ac7e7cce63, UNASSIGN in 178 msec 2023-02-09 16:55:53,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:53,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=b37c49ed7e31887c6eb48358966d0208, regionState=CLOSED 2023-02-09 16:55:53,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675961753046"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961753046"}]},"ts":"1675961753046"} 2023-02-09 16:55:53,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-09 16:55:53,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure b37c49ed7e31887c6eb48358966d0208, server=jenkins-hbase4.apache.org,43347,1675961647698 in 177 msec 2023-02-09 16:55:53,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-09 16:55:53,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=b37c49ed7e31887c6eb48358966d0208, UNASSIGN in 182 msec 2023-02-09 16:55:53,051 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961753050"}]},"ts":"1675961753050"} 2023-02-09 16:55:53,051 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-09 16:55:53,053 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-09 16:55:53,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 191 msec 2023-02-09 16:55:53,166 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,166 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 16:55:53,167 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-09 16:55:53,167 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,167 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,167 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,170 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,170 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,170 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,170 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 16:55:53,174 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 2023-02-09 16:55:53,174 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:53,174 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:53,176 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/recovered.edits] 2023-02-09 16:55:53,176 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/recovered.edits] 2023-02-09 16:55:53,176 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/recovered.edits] 2023-02-09 16:55:53,179 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1a8d884b02134330b8c304bc34c23813 2023-02-09 16:55:53,179 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/a3741ae28dc844f78309064a6fa7075e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf1/a3741ae28dc844f78309064a6fa7075e 2023-02-09 16:55:53,180 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/1bd1d7eef8be43439117973506434bb6 2023-02-09 16:55:53,181 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/recovered.edits/88.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208/recovered.edits/88.seqid 2023-02-09 16:55:53,181 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/4cccf561ba9b4cbebadbb372aed8e186 2023-02-09 16:55:53,181 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/90e71d787b4b4021be4ba60446091160 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/cf2/90e71d787b4b4021be4ba60446091160 2023-02-09 16:55:53,182 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/b37c49ed7e31887c6eb48358966d0208 2023-02-09 16:55:53,182 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/564978c7573b4208aafddc5f81717507 2023-02-09 16:55:53,183 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/792d518eafda4b999bcb0cbb1cc54531 2023-02-09 16:55:53,183 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63/recovered.edits/90.seqid 2023-02-09 16:55:53,184 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b00796ea92440e745d6d3ac7e7cce63 2023-02-09 16:55:53,184 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/8cb74e31110245a3a7cc29fb2042def6 2023-02-09 16:55:53,185 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf1/a646ac64bc974db78d4209dd62a63e6a 2023-02-09 16:55:53,187 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/551378437769498e8a633a8d6a54a7b7 2023-02-09 16:55:53,188 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/61aecb055b4b40cdadb0d60bd1f6daab 2023-02-09 16:55:53,188 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/772053991fe540da82b38afb1593c49a 2023-02-09 16:55:53,189 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/88b52ab9c43c4bcbbeaa4170927016be 2023-02-09 16:55:53,190 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/98952011c2ad4b37b3aa7bef176ae0c6 2023-02-09 16:55:53,191 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/e1cd007bcfdb4160b8a06c6ca1454158 2023-02-09 16:55:53,192 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/cf2/fb4a56f7896e4ff295dea9f7a3671e9e 2023-02-09 16:55:53,193 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/recovered.edits/85.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170/recovered.edits/85.seqid 2023-02-09 16:55:53,194 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegionEmptyDaughter/e223b141ea1d973993597987c0d85170 2023-02-09 16:55:53,194 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 16:55:53,195 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,197 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-09 16:55:53,199 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-09 16:55:53,200 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,200 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-09 16:55:53,200 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961753200"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:53,200 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961753200"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:53,200 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961753200"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:53,201 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:55:53,201 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e223b141ea1d973993597987c0d85170, NAME => 'testSplitTableRegionEmptyDaughter,,1675961749270.e223b141ea1d973993597987c0d85170.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 9b00796ea92440e745d6d3ac7e7cce63, NAME => 'testSplitTableRegionEmptyDaughter,,1675961751994.9b00796ea92440e745d6d3ac7e7cce63.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => b37c49ed7e31887c6eb48358966d0208, NAME => 'testSplitTableRegionEmptyDaughter,71,1675961751994.b37c49ed7e31887c6eb48358966d0208.', STARTKEY => '71', ENDKEY => ''}] 2023-02-09 16:55:53,201 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-09 16:55:53,201 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961753201"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:53,203 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-09 16:55:53,204 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 16:55:53,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 36 msec 2023-02-09 16:55:53,271 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,271 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 16:55:53,272 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-09 16:55:53,285 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374 (was 376), OpenFileDescriptor=823 (was 825), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=111 (was 95) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=2900 (was 2901) 2023-02-09 16:55:53,299 INFO [Listener at localhost/39015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374, OpenFileDescriptor=823, MaxFileDescriptor=60000, SystemLoadAverage=111, ProcessCount=168, AvailableMemoryMB=2899 2023-02-09 16:55:53,299 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:53,299 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,300 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,303 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 16:55:53,303 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 16:55:53,304 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-09 16:55:53,305 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 16:55:53,306 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 16:55:53,307 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,307 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 empty. 2023-02-09 16:55:53,308 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,308 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 16:55:53,320 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-09 16:55:53,321 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 981cad13cbee8b41b5ca0150bff7cc45, NAME => 'testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp 2023-02-09 16:55:53,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:53,327 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 981cad13cbee8b41b5ca0150bff7cc45, disabling compactions & flushes 2023-02-09 16:55:53,327 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,328 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,328 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. after waiting 0 ms 2023-02-09 16:55:53,328 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,328 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,328 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 981cad13cbee8b41b5ca0150bff7cc45: Waiting for close lock at 1675961753327Disabling compacts and flushes for region at 1675961753327Disabling writes for close at 1675961753328 (+1 ms)Writing region close event to WAL at 1675961753328Closed at 1675961753328 2023-02-09 16:55:53,329 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 16:55:53,330 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961753330"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961753330"}]},"ts":"1675961753330"} 2023-02-09 16:55:53,331 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 16:55:53,332 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 16:55:53,332 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961753332"}]},"ts":"1675961753332"} 2023-02-09 16:55:53,332 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-09 16:55:53,335 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 16:55:53,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 16:55:53,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 16:55:53,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 16:55:53,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 16:55:53,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, ASSIGN}] 2023-02-09 16:55:53,337 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, ASSIGN 2023-02-09 16:55:53,337 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:55:53,488 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 16:55:53,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=981cad13cbee8b41b5ca0150bff7cc45, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961753489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961753489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961753489"}]},"ts":"1675961753489"} 2023-02-09 16:55:53,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 981cad13cbee8b41b5ca0150bff7cc45, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:53,577 ERROR [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid 627681a5-99ed-41c7-b973-a8e3d411bd34) service to localhost/127.0.0.1:44699 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 16:55:53,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 981cad13cbee8b41b5ca0150bff7cc45, NAME => 'testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.', STARTKEY => '', ENDKEY => ''} 2023-02-09 16:55:53,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:53,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:53,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,646 INFO [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,647 DEBUG [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1 2023-02-09 16:55:53,647 DEBUG [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1 2023-02-09 16:55:53,647 INFO [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 981cad13cbee8b41b5ca0150bff7cc45 columnFamilyName cf1 2023-02-09 16:55:53,648 INFO [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] regionserver.HStore(310): Store=981cad13cbee8b41b5ca0150bff7cc45/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:53,648 INFO [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,649 DEBUG [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2 2023-02-09 16:55:53,649 DEBUG [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2 2023-02-09 16:55:53,649 INFO [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 981cad13cbee8b41b5ca0150bff7cc45 columnFamilyName cf2 2023-02-09 16:55:53,650 INFO [StoreOpener-981cad13cbee8b41b5ca0150bff7cc45-1] regionserver.HStore(310): Store=981cad13cbee8b41b5ca0150bff7cc45/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:53,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:53,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:53,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 16:55:53,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 981cad13cbee8b41b5ca0150bff7cc45; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63422567, jitterRate=-0.054930105805397034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:53,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 981cad13cbee8b41b5ca0150bff7cc45: Running coprocessor pre-open hook at 1675961753645Writing region info on filesystem at 1675961753645Initializing all the Stores at 1675961753645Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961753645Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961753646 (+1 ms)Cleaning up temporary data from old regions at 1675961753651 (+5 ms)Cleaning up detritus from prior splits at 1675961753651Running coprocessor post-open hooks at 1675961753654 (+3 ms)Region opened successfully at 1675961753654 2023-02-09 16:55:53,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45., pid=143, masterSystemTime=1675961753642 2023-02-09 16:55:53,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,656 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=981cad13cbee8b41b5ca0150bff7cc45, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961753656"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961753656"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961753656"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961753656"}]},"ts":"1675961753656"} 2023-02-09 16:55:53,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-09 16:55:53,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 981cad13cbee8b41b5ca0150bff7cc45, server=jenkins-hbase4.apache.org,43347,1675961647698 in 167 msec 2023-02-09 16:55:53,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-09 16:55:53,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, ASSIGN in 323 msec 2023-02-09 16:55:53,660 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 16:55:53,660 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961753660"}]},"ts":"1675961753660"} 2023-02-09 16:55:53,661 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-09 16:55:53,663 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 16:55:53,664 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 359 msec 2023-02-09 16:55:53,811 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,811 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,811 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:53,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 16:55:53,813 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 16:55:53,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 16:55:53,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 16:55:53,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,813 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,815 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,815 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,815 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,815 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,816 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,816 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:53,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:53,818 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:53,818 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,817 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:53,818 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:53,818 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. started... 2023-02-09 16:55:53,819 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 16:55:53,819 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,819 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,820 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,820 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,820 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:53,820 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:53,827 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/b14cb51616014d4dba86966f459f16e9 2023-02-09 16:55:53,837 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/b841fc2288e7431f8b2e95da5920d4fe 2023-02-09 16:55:53,841 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/b14cb51616014d4dba86966f459f16e9 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9 2023-02-09 16:55:53,844 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:53,845 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/b841fc2288e7431f8b2e95da5920d4fe as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe 2023-02-09 16:55:53,847 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:53,848 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 29ms, sequenceid=6, compaction requested=false 2023-02-09 16:55:53,848 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-09 16:55:53,849 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 981cad13cbee8b41b5ca0150bff7cc45: 2023-02-09 16:55:53,849 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,849 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:53,849 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:53,849 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,849 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:53,849 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:53,850 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,850 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:53,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:53,850 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,851 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:53,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 16:55:53,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69ff7fab[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:53,852 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 16:55:53,852 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,853 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:53,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:53,853 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:53,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:53,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,853 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:53,853 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:53,854 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:53,854 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:53,853 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:53,854 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:53,855 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,855 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:53,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:53,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:53,855 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:53,855 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:53,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:53,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:53,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:53,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:53,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:53,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:53,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 16:55:53,864 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 16:55:53,864 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:53,864 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:53,864 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 16:55:53,864 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:53,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:53,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,865 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:53,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:53,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:53,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:53,866 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:53,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:53,867 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:53,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:53,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,867 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:53,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:53,869 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,869 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,870 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,871 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 16:55:53,872 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:53,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:53,872 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:53,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:53,872 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,873 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:53,873 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:53,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:53,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:53,873 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:53,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:53,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:53,873 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,873 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,973 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:53,973 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,973 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:53,986 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:53,986 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:53,987 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:53,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:53,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 16:55:53,990 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 16:55:53,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 16:55:53,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 16:55:53,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,990 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,991 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,991 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,991 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,991 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:53,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:53,992 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:53,992 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:53,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:53,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:53,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:53,993 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:53,994 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:53,994 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:53,994 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:53,994 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. started... 2023-02-09 16:55:53,994 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:53,996 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,997 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,997 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,997 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:53,997 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:53,997 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,008 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/02f3f3de62244b1dbad87ea070d1dbbf 2023-02-09 16:55:54,422 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/dfe70c05ebd2404985f196f8e25e1660 2023-02-09 16:55:54,426 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/02f3f3de62244b1dbad87ea070d1dbbf as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf 2023-02-09 16:55:54,429 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:54,430 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/dfe70c05ebd2404985f196f8e25e1660 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660 2023-02-09 16:55:54,433 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 16:55:54,434 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 440ms, sequenceid=19, compaction requested=false 2023-02-09 16:55:54,434 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 981cad13cbee8b41b5ca0150bff7cc45: 2023-02-09 16:55:54,434 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,434 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:54,434 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:54,434 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,434 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,434 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,435 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 16:55:54,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30333fc1[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:54,437 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 16:55:54,438 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,439 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,439 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,442 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,442 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:54,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 16:55:54,445 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 16:55:54,445 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:54,445 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:54,445 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 16:55:54,445 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:54,446 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,446 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,446 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,447 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,447 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,447 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,448 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,449 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,449 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,449 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,452 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 16:55:54,452 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,452 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:54,452 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:54,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,453 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,553 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:54,553 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:54,553 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:54,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:54,565 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:54,565 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:54,566 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:54,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:54,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 16:55:54,567 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 16:55:54,567 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,568 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 16:55:54,568 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 16:55:54,568 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,568 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,569 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,569 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,569 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,569 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,569 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,569 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:54,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:54,571 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:54,571 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,572 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. started... 2023-02-09 16:55:54,572 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:54,573 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,576 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,576 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,576 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,577 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,577 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,580 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/0e2004e6754340fcb512c4d7fb434ab2 2023-02-09 16:55:54,589 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/4101c9f4a5214361a03a913eb094ced7 2023-02-09 16:55:54,593 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/0e2004e6754340fcb512c4d7fb434ab2 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2 2023-02-09 16:55:54,595 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:54,596 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/4101c9f4a5214361a03a913eb094ced7 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7 2023-02-09 16:55:54,598 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 16:55:54,599 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 27ms, sequenceid=32, compaction requested=true 2023-02-09 16:55:54,599 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 981cad13cbee8b41b5ca0150bff7cc45: 2023-02-09 16:55:54,599 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,599 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,600 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,600 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,601 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,601 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 16:55:54,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62836de[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:54,602 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 16:55:54,602 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,604 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,604 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,604 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,604 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:54,605 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,605 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,605 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:54,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,605 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,605 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,605 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:54,606 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,606 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,606 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,606 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,606 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,606 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,606 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:54,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 16:55:54,609 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 16:55:54,609 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:54,609 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:54,609 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 16:55:54,609 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,612 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,613 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,613 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,615 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,615 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,615 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,617 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 16:55:54,618 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:54,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,617 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:54,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,618 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,718 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:54,718 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:54,718 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:54,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:54,729 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:54,729 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:54,729 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:54,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:54,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 16:55:54,731 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 16:55:54,731 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,731 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 16:55:54,731 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 16:55:54,731 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,731 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,732 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,732 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,732 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,732 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,733 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,733 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,735 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,735 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,735 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:54,735 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,735 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,735 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. started... 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,735 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,735 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:54,736 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,738 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,738 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,738 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,739 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,739 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,746 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/55d2bcbb00874f58a2d901119529d79e 2023-02-09 16:55:54,756 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/06656ed91c0e47dfbe6256e36ad731a8 2023-02-09 16:55:54,760 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/55d2bcbb00874f58a2d901119529d79e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e 2023-02-09 16:55:54,763 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:54,763 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/06656ed91c0e47dfbe6256e36ad731a8 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8 2023-02-09 16:55:54,766 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 16:55:54,767 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 32ms, sequenceid=45, compaction requested=true 2023-02-09 16:55:54,767 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 981cad13cbee8b41b5ca0150bff7cc45: 2023-02-09 16:55:54,767 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,767 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:54,767 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:54,767 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,767 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,767 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,769 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,769 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 16:55:54,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55cf2176[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:54,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 16:55:54,771 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,772 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,772 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,772 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,772 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:54,773 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,773 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,773 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,775 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,775 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,776 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,776 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,776 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,776 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,776 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 16:55:54,778 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 16:55:54,779 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:54,779 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:54,779 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 16:55:54,779 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,780 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,781 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,782 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,782 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,784 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,784 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,784 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,793 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,793 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,793 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,793 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,793 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,793 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,794 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 16:55:54,794 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:54,794 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,794 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,795 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,895 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:54,895 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:54,895 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:54,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:54,904 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:54,904 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:54,905 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:54,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:54,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 16:55:54,906 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 16:55:54,906 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,906 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 16:55:54,906 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 16:55:54,906 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,907 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,909 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,909 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,909 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,909 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,909 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,909 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:54,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:54,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:54,911 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:54,912 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,912 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. started... 2023-02-09 16:55:54,912 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:54,912 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,913 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,920 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/64ec367f30854c1cb95f513ad1926d0a 2023-02-09 16:55:54,929 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 2023-02-09 16:55:54,932 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/64ec367f30854c1cb95f513ad1926d0a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a 2023-02-09 16:55:54,935 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:54,936 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 2023-02-09 16:55:54,939 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 16:55:54,939 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 27ms, sequenceid=58, compaction requested=true 2023-02-09 16:55:54,939 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 981cad13cbee8b41b5ca0150bff7cc45: 2023-02-09 16:55:54,939 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:54,939 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:54,939 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:54,939 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:54,941 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 16:55:54,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47df1e43[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:54,943 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 16:55:54,943 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,945 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,945 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:54,945 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,945 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:54,946 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,947 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,947 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 16:55:54,950 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 16:55:54,950 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:54,950 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:54,950 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 16:55:54,950 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,951 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:54,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,952 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,953 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:54,953 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:54,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:54,954 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,955 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,955 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,957 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 16:55:54,957 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:54,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,957 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,957 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:54,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:54,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:54,958 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:54,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:54,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,058 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:55,058 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:55,058 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:55,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:55,069 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:55,069 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:55,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 16:55:55,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 16:55:55,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 16:55:55,071 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 16:55:55,071 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:55,071 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 16:55:55,071 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 16:55:55,071 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,072 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,074 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:55,074 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:55,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:55,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:55,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:55,074 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:55,074 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:55,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:55,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:55,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,074 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,074 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 16:55:55,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:55,075 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:55,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 16:55:55,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:55,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37073,1675961647745' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:55,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:55,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:55,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:55,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 16:55:55,076 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33383,1675961647775' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 16:55:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 16:55:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 16:55:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 16:55:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 16:55:55,077 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:55,077 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. started... 2023-02-09 16:55:55,077 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 16:55:55,077 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,079 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,079 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,079 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:55,080 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,080 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:55,090 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/28040a6d692d4e9d9035d409d2419014 2023-02-09 16:55:55,101 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/e2b41d6fe4bd4f6b97caeb019402509a 2023-02-09 16:55:55,104 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/28040a6d692d4e9d9035d409d2419014 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014 2023-02-09 16:55:55,107 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:55,107 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/e2b41d6fe4bd4f6b97caeb019402509a as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a 2023-02-09 16:55:55,111 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 16:55:55,111 INFO [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 34ms, sequenceid=71, compaction requested=true 2023-02-09 16:55:55,111 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 981cad13cbee8b41b5ca0150bff7cc45: 2023-02-09 16:55:55,111 DEBUG [rs(jenkins-hbase4.apache.org,43347,1675961647698)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:55,111 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 16:55:55,111 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 16:55:55,111 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,112 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 16:55:55,112 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 16:55:55,113 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,113 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:55,113 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,113 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 16:55:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:55,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43347,1675961647698' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 16:55:55,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e3bfc4e[Count = 0] remaining members to acquire global barrier 2023-02-09 16:55:55,115 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 16:55:55,115 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:55,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37073,1675961647745' in zk 2023-02-09 16:55:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:55,117 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,117 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 16:55:55,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:55,117 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43347,1675961647698' in zk 2023-02-09 16:55:55,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,118 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 16:55:55,118 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 16:55:55,118 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33383,1675961647775' in zk 2023-02-09 16:55:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,37073,1675961647745' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:55,121 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,121 DEBUG [member: 'jenkins-hbase4.apache.org,43347,1675961647698' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:55,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:55,123 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 16:55:55,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:55,123 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:55,123 DEBUG [member: 'jenkins-hbase4.apache.org,33383,1675961647775' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 16:55:55,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43347,1675961647698': 2023-02-09 16:55:55,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43347,1675961647698' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 16:55:55,126 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 16:55:55,126 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 16:55:55,126 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 16:55:55,126 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 16:55:55,126 INFO [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 16:55:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 16:55:55,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 16:55:55,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,127 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:55,128 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:55,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:55,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,128 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,128 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 16:55:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 16:55:55,130 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,130 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,130 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:55,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 16:55:55,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:55,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:55,133 DEBUG [(jenkins-hbase4.apache.org,35255,1675961646540)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 16:55:55,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 16:55:55,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,133 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 16:55:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 16:55:55,134 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 16:55:55,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 16:55:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 16:55:55,234 DEBUG [Listener at localhost/39015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 16:55:55,234 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:55,234 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:55,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 16:55:55,242 DEBUG [Listener at localhost/39015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,242 DEBUG [Listener at localhost/39015] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=981cad13cbee8b41b5ca0150bff7cc45, daughterA=30ac7931893e54415df2b3d07a28d96c, daughterB=12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:55,244 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=981cad13cbee8b41b5ca0150bff7cc45, daughterA=30ac7931893e54415df2b3d07a28d96c, daughterB=12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:55,244 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=981cad13cbee8b41b5ca0150bff7cc45, daughterA=30ac7931893e54415df2b3d07a28d96c, daughterB=12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:55,244 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=981cad13cbee8b41b5ca0150bff7cc45, daughterA=30ac7931893e54415df2b3d07a28d96c, daughterB=12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:55,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, UNASSIGN}] 2023-02-09 16:55:55,245 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, UNASSIGN 2023-02-09 16:55:55,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=981cad13cbee8b41b5ca0150bff7cc45, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,246 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961755246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961755246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961755246"}]},"ts":"1675961755246"} 2023-02-09 16:55:55,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 981cad13cbee8b41b5ca0150bff7cc45, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:55,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 981cad13cbee8b41b5ca0150bff7cc45, disabling compactions & flushes 2023-02-09 16:55:55,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:55,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:55,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. after waiting 0 ms 2023-02-09 16:55:55,399 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:55,399 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 981cad13cbee8b41b5ca0150bff7cc45 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 16:55:55,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/7748adcc31ab4d88a0acd2b63745158c 2023-02-09 16:55:55,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 2023-02-09 16:55:55,434 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf1/7748adcc31ab4d88a0acd2b63745158c as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c 2023-02-09 16:55:55,437 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:55,437 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/.tmp/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 2023-02-09 16:55:55,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 16:55:55,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 42ms, sequenceid=82, compaction requested=true 2023-02-09 16:55:55,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 16:55:55,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:55,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. 2023-02-09 16:55:55,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 981cad13cbee8b41b5ca0150bff7cc45: Waiting for close lock at 1675961755399Running coprocessor pre-close hooks at 1675961755399Disabling compacts and flushes for region at 1675961755399Disabling writes for close at 1675961755399Obtaining lock to block concurrent updates at 1675961755399Preparing flush snapshotting stores in 981cad13cbee8b41b5ca0150bff7cc45 at 1675961755399Finished memstore snapshotting testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675961755399Flushing stores of testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. at 1675961755400 (+1 ms)Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf1: creating writer at 1675961755401 (+1 ms)Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf1: appending metadata at 1675961755403 (+2 ms)Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf1: closing flushed file at 1675961755403Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf2: creating writer at 1675961755421 (+18 ms)Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf2: appending metadata at 1675961755422 (+1 ms)Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf2: closing flushed file at 1675961755422Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf1: reopening flushed file at 1675961755434 (+12 ms)Flushing 981cad13cbee8b41b5ca0150bff7cc45/cf2: reopening flushed file at 1675961755438 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 981cad13cbee8b41b5ca0150bff7cc45 in 42ms, sequenceid=82, compaction requested=true at 1675961755441 (+3 ms)Writing region close event to WAL at 1675961755444 (+3 ms)Running coprocessor post-close hooks at 1675961755447 (+3 ms)Closed at 1675961755447 2023-02-09 16:55:55,448 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=981cad13cbee8b41b5ca0150bff7cc45, regionState=CLOSED 2023-02-09 16:55:55,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961755448"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961755448"}]},"ts":"1675961755448"} 2023-02-09 16:55:55,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-09 16:55:55,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 981cad13cbee8b41b5ca0150bff7cc45, server=jenkins-hbase4.apache.org,43347,1675961647698 in 202 msec 2023-02-09 16:55:55,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-09 16:55:55,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=981cad13cbee8b41b5ca0150bff7cc45, UNASSIGN in 206 msec 2023-02-09 16:55:55,460 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=981cad13cbee8b41b5ca0150bff7cc45, threads=14 2023-02-09 16:55:55,460 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,460 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,461 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,462 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,462 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,462 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,462 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,592 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,600 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,600 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,608 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,620 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,630 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,630 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,630 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,632 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,636 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,640 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,649 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,650 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,650 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7 for region: 981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:55,650 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 981cad13cbee8b41b5ca0150bff7cc45 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 16:55:55,690 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:55:55,691 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 16:55:55,692 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675961755692"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675961755692"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675961755692"}]},"ts":"1675961755692"} 2023-02-09 16:55:55,692 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961755692"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961755692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961755692"}]},"ts":"1675961755692"} 2023-02-09 16:55:55,693 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961755692"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961755692"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961755692"}]},"ts":"1675961755692"} 2023-02-09 16:55:55,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, ASSIGN}] 2023-02-09 16:55:55,696 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, ASSIGN 2023-02-09 16:55:55,696 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:55:55,697 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, ASSIGN 2023-02-09 16:55:55,697 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43347,1675961647698; forceNewPlan=false, retain=false 2023-02-09 16:55:55,846 INFO [jenkins-hbase4:35255] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 16:55:55,848 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=30ac7931893e54415df2b3d07a28d96c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,849 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961755848"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961755848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961755848"}]},"ts":"1675961755848"} 2023-02-09 16:55:55,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 30ac7931893e54415df2b3d07a28d96c, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:55,852 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=12c9b49df55b6017a6b7a81fa449a545, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:55,852 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961755852"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961755852"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961755852"}]},"ts":"1675961755852"} 2023-02-09 16:55:55,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 12c9b49df55b6017a6b7a81fa449a545, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:56,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 30ac7931893e54415df2b3d07a28d96c, NAME => 'testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 16:55:56,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:56,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:56,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,005 INFO [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,006 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1 2023-02-09 16:55:56,006 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1 2023-02-09 16:55:56,007 INFO [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30ac7931893e54415df2b3d07a28d96c columnFamilyName cf1 2023-02-09 16:55:56,019 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf-bottom 2023-02-09 16:55:56,022 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2-bottom 2023-02-09 16:55:56,025 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014-bottom 2023-02-09 16:55:56,029 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e-bottom 2023-02-09 16:55:56,032 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a-bottom 2023-02-09 16:55:56,036 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c-bottom 2023-02-09 16:55:56,039 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9-bottom 2023-02-09 16:55:56,039 INFO [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(310): Store=30ac7931893e54415df2b3d07a28d96c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:56,039 INFO [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,040 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2 2023-02-09 16:55:56,040 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2 2023-02-09 16:55:56,040 INFO [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30ac7931893e54415df2b3d07a28d96c columnFamilyName cf2 2023-02-09 16:55:56,051 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8-bottom 2023-02-09 16:55:56,055 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668-bottom 2023-02-09 16:55:56,058 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7-bottom 2023-02-09 16:55:56,062 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe-bottom 2023-02-09 16:55:56,065 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5-bottom 2023-02-09 16:55:56,069 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660-bottom 2023-02-09 16:55:56,072 DEBUG [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a-bottom 2023-02-09 16:55:56,072 INFO [StoreOpener-30ac7931893e54415df2b3d07a28d96c-1] regionserver.HStore(310): Store=30ac7931893e54415df2b3d07a28d96c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:56,073 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:56,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 30ac7931893e54415df2b3d07a28d96c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68550448, jitterRate=0.02148127555847168}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:56,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 30ac7931893e54415df2b3d07a28d96c: Running coprocessor pre-open hook at 1675961756005Writing region info on filesystem at 1675961756005Initializing all the Stores at 1675961756005Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961756005Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961756005Cleaning up temporary data from old regions at 1675961756074 (+69 ms)Cleaning up detritus from prior splits at 1675961756075 (+1 ms)Running coprocessor post-open hooks at 1675961756077 (+2 ms)Region opened successfully at 1675961756077 2023-02-09 16:55:56,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c., pid=149, masterSystemTime=1675961756001 2023-02-09 16:55:56,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 16:55:56,079 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:56,079 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,079 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:56,079 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): 30ac7931893e54415df2b3d07a28d96c/cf1 is initiating minor compaction (all files) 2023-02-09 16:55:56,079 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,079 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 30ac7931893e54415df2b3d07a28d96c/cf1 in testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,079 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): 30ac7931893e54415df2b3d07a28d96c/cf2 is initiating minor compaction (all files) 2023-02-09 16:55:56,079 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 30ac7931893e54415df2b3d07a28d96c/cf2 in testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,079 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/.tmp, totalSize=34.9 K 2023-02-09 16:55:56,080 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668-bottom] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/.tmp, totalSize=34.9 K 2023-02-09 16:55:56,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,080 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961753807 2023-02-09 16:55:56,080 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675961753807 2023-02-09 16:55:56,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 12c9b49df55b6017a6b7a81fa449a545, NAME => 'testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 16:55:56,080 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=30ac7931893e54415df2b3d07a28d96c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:56,080 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961753974 2023-02-09 16:55:56,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 16:55:56,081 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675961753974 2023-02-09 16:55:56,080 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961756080"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961756080"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961756080"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961756080"}]},"ts":"1675961756080"} 2023-02-09 16:55:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 16:55:56,081 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961754555 2023-02-09 16:55:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,081 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675961754555 2023-02-09 16:55:56,081 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961754720 2023-02-09 16:55:56,081 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675961754720 2023-02-09 16:55:56,082 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961754896 2023-02-09 16:55:56,082 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675961754896 2023-02-09 16:55:56,082 INFO [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,082 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961755060 2023-02-09 16:55:56,083 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675961755060 2023-02-09 16:55:56,083 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961755236 2023-02-09 16:55:56,083 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1 2023-02-09 16:55:56,083 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1 2023-02-09 16:55:56,083 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675961755236 2023-02-09 16:55:56,084 INFO [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12c9b49df55b6017a6b7a81fa449a545 columnFamilyName cf1 2023-02-09 16:55:56,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-09 16:55:56,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 30ac7931893e54415df2b3d07a28d96c, server=jenkins-hbase4.apache.org,43347,1675961647698 in 232 msec 2023-02-09 16:55:56,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, ASSIGN in 390 msec 2023-02-09 16:55:56,094 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): 30ac7931893e54415df2b3d07a28d96c#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:56,096 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): 30ac7931893e54415df2b3d07a28d96c#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:56,103 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf-top 2023-02-09 16:55:56,106 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2-top 2023-02-09 16:55:56,116 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014-top 2023-02-09 16:55:56,129 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e-top 2023-02-09 16:55:56,132 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/.tmp/cf2/1d9832b9a2af48cb98c9f8bf11b9434e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/1d9832b9a2af48cb98c9f8bf11b9434e 2023-02-09 16:55:56,133 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/.tmp/cf1/b137253ac21b4ece98808c35d4f397ca as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b137253ac21b4ece98808c35d4f397ca 2023-02-09 16:55:56,133 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a-top 2023-02-09 16:55:56,136 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c-top 2023-02-09 16:55:56,137 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 30ac7931893e54415df2b3d07a28d96c/cf2 of 30ac7931893e54415df2b3d07a28d96c into 1d9832b9a2af48cb98c9f8bf11b9434e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:56,137 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 30ac7931893e54415df2b3d07a28d96c: 2023-02-09 16:55:56,137 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c., storeName=30ac7931893e54415df2b3d07a28d96c/cf2, priority=9, startTime=1675961756079; duration=0sec 2023-02-09 16:55:56,137 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,138 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 30ac7931893e54415df2b3d07a28d96c/cf1 of 30ac7931893e54415df2b3d07a28d96c into b137253ac21b4ece98808c35d4f397ca(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:56,138 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 30ac7931893e54415df2b3d07a28d96c: 2023-02-09 16:55:56,138 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c., storeName=30ac7931893e54415df2b3d07a28d96c/cf1, priority=9, startTime=1675961756078; duration=0sec 2023-02-09 16:55:56,138 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,140 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9-top 2023-02-09 16:55:56,140 INFO [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(310): Store=12c9b49df55b6017a6b7a81fa449a545/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:56,140 INFO [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,141 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2 2023-02-09 16:55:56,141 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2 2023-02-09 16:55:56,141 INFO [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12c9b49df55b6017a6b7a81fa449a545 columnFamilyName cf2 2023-02-09 16:55:56,153 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8-top 2023-02-09 16:55:56,155 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668-top 2023-02-09 16:55:56,158 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7-top 2023-02-09 16:55:56,161 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe-top 2023-02-09 16:55:56,168 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5-top 2023-02-09 16:55:56,172 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660-top 2023-02-09 16:55:56,175 DEBUG [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(539): loaded hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a-top 2023-02-09 16:55:56,175 INFO [StoreOpener-12c9b49df55b6017a6b7a81fa449a545-1] regionserver.HStore(310): Store=12c9b49df55b6017a6b7a81fa449a545/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 16:55:56,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 16:55:56,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 12c9b49df55b6017a6b7a81fa449a545; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70073385, jitterRate=0.04417480528354645}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 16:55:56,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 12c9b49df55b6017a6b7a81fa449a545: Running coprocessor pre-open hook at 1675961756081Writing region info on filesystem at 1675961756081Initializing all the Stores at 1675961756082 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961756082Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675961756082Cleaning up temporary data from old regions at 1675961756177 (+95 ms)Cleaning up detritus from prior splits at 1675961756178 (+1 ms)Running coprocessor post-open hooks at 1675961756179 (+1 ms)Region opened successfully at 1675961756179 2023-02-09 16:55:56,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545., pid=150, masterSystemTime=1675961756001 2023-02-09 16:55:56,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,181 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:56,181 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 16:55:56,181 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,181 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1912): 12c9b49df55b6017a6b7a81fa449a545/cf2 is initiating minor compaction (all files) 2023-02-09 16:55:56,181 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,182 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 12c9b49df55b6017a6b7a81fa449a545/cf2 in testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,182 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1912): 12c9b49df55b6017a6b7a81fa449a545/cf1 is initiating minor compaction (all files) 2023-02-09 16:55:56,182 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 12c9b49df55b6017a6b7a81fa449a545/cf1 in testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,182 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668-top] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/.tmp, totalSize=34.9 K 2023-02-09 16:55:56,182 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c-top] into tmpdir=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/.tmp, totalSize=34.9 K 2023-02-09 16:55:56,182 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961753807 2023-02-09 16:55:56,183 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675961753807 2023-02-09 16:55:56,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,183 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=12c9b49df55b6017a6b7a81fa449a545, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:56,183 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961753974 2023-02-09 16:55:56,183 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961756183"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675961756183"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675961756183"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675961756183"}]},"ts":"1675961756183"} 2023-02-09 16:55:56,184 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675961753974 2023-02-09 16:55:56,184 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961754555 2023-02-09 16:55:56,184 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675961754555 2023-02-09 16:55:56,184 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961754720 2023-02-09 16:55:56,184 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675961754720 2023-02-09 16:55:56,185 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675961754896 2023-02-09 16:55:56,185 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675961755060 2023-02-09 16:55:56,185 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675961754896 2023-02-09 16:55:56,185 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] compactions.Compactor(207): Compacting 175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961755236 2023-02-09 16:55:56,186 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675961755060 2023-02-09 16:55:56,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-09 16:55:56,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 12c9b49df55b6017a6b7a81fa449a545, server=jenkins-hbase4.apache.org,43347,1675961647698 in 331 msec 2023-02-09 16:55:56,186 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] compactions.Compactor(207): Compacting 7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675961755236 2023-02-09 16:55:56,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-09 16:55:56,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, ASSIGN in 491 msec 2023-02-09 16:55:56,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=981cad13cbee8b41b5ca0150bff7cc45, daughterA=30ac7931893e54415df2b3d07a28d96c, daughterB=12c9b49df55b6017a6b7a81fa449a545 in 947 msec 2023-02-09 16:55:56,196 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] throttle.PressureAwareThroughputController(145): 12c9b49df55b6017a6b7a81fa449a545#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:56,196 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] throttle.PressureAwareThroughputController(145): 12c9b49df55b6017a6b7a81fa449a545#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 16:55:56,214 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/.tmp/cf2/0f6312332fd34b0699a4c8d2417949f0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/0f6312332fd34b0699a4c8d2417949f0 2023-02-09 16:55:56,218 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/.tmp/cf1/2bd32efed4b64fc2ad0a96ddf24ad376 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/2bd32efed4b64fc2ad0a96ddf24ad376 2023-02-09 16:55:56,219 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 12c9b49df55b6017a6b7a81fa449a545/cf2 of 12c9b49df55b6017a6b7a81fa449a545 into 0f6312332fd34b0699a4c8d2417949f0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:56,219 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 12c9b49df55b6017a6b7a81fa449a545: 2023-02-09 16:55:56,219 INFO [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545., storeName=12c9b49df55b6017a6b7a81fa449a545/cf2, priority=9, startTime=1675961756180; duration=0sec 2023-02-09 16:55:56,219 DEBUG [RS:0;jenkins-hbase4:43347-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,222 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 12c9b49df55b6017a6b7a81fa449a545/cf1 of 12c9b49df55b6017a6b7a81fa449a545 into 2bd32efed4b64fc2ad0a96ddf24ad376(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 16:55:56,222 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 12c9b49df55b6017a6b7a81fa449a545: 2023-02-09 16:55:56,222 INFO [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545., storeName=12c9b49df55b6017a6b7a81fa449a545/cf1, priority=9, startTime=1675961756180; duration=0sec 2023-02-09 16:55:56,222 DEBUG [RS:0;jenkins-hbase4:43347-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 16:55:56,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43347] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:36868 deadline: 1675961816244, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45. is not online on jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:56,354 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:55:56,360 INFO [Listener at localhost/39015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 16:55:56,361 DEBUG [Listener at localhost/39015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 16:55:56,367 WARN [Listener at localhost/39015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 16:55:56,367 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,367 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,368 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,368 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,368 INFO [Listener at localhost/39015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-09 16:55:56,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-09 16:55:56,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,371 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,371 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 16:55:56,372 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961756372"}]},"ts":"1675961756372"} 2023-02-09 16:55:56,373 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-09 16:55:56,375 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-09 16:55:56,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, UNASSIGN}] 2023-02-09 16:55:56,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, UNASSIGN 2023-02-09 16:55:56,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=12c9b49df55b6017a6b7a81fa449a545, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:56,376 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961756376"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961756376"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961756376"}]},"ts":"1675961756376"} 2023-02-09 16:55:56,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 12c9b49df55b6017a6b7a81fa449a545, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:56,379 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, UNASSIGN 2023-02-09 16:55:56,379 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=30ac7931893e54415df2b3d07a28d96c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:56,379 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961756379"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675961756379"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675961756379"}]},"ts":"1675961756379"} 2023-02-09 16:55:56,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 30ac7931893e54415df2b3d07a28d96c, server=jenkins-hbase4.apache.org,43347,1675961647698}] 2023-02-09 16:55:56,472 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,472 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 16:55:56,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 30ac7931893e54415df2b3d07a28d96c, disabling compactions & flushes 2023-02-09 16:55:56,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. after waiting 0 ms 2023-02-09 16:55:56,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,530 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c-bottom] to archive 2023-02-09 16:55:56,531 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:56,532 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,533 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,534 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,535 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,536 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,537 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,538 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,539 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a-bottom, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668-bottom] to archive 2023-02-09 16:55:56,540 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:56,541 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,542 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,543 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,544 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,545 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,546 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,546 DEBUG [StoreCloser-testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:55:56,550 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:56,550 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c. 2023-02-09 16:55:56,550 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 30ac7931893e54415df2b3d07a28d96c: Waiting for close lock at 1675961756529Running coprocessor pre-close hooks at 1675961756529Disabling compacts and flushes for region at 1675961756529Disabling writes for close at 1675961756529Writing region close event to WAL at 1675961756547 (+18 ms)Running coprocessor post-close hooks at 1675961756550 (+3 ms)Closed at 1675961756550 2023-02-09 16:55:56,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 12c9b49df55b6017a6b7a81fa449a545, disabling compactions & flushes 2023-02-09 16:55:56,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. after waiting 0 ms 2023-02-09 16:55:56,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=30ac7931893e54415df2b3d07a28d96c, regionState=CLOSED 2023-02-09 16:55:56,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961756551"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961756551"}]},"ts":"1675961756551"} 2023-02-09 16:55:56,552 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c-top] to archive 2023-02-09 16:55:56,553 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:56,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-09 16:55:56,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 30ac7931893e54415df2b3d07a28d96c, server=jenkins-hbase4.apache.org,43347,1675961647698 in 173 msec 2023-02-09 16:55:56,554 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/b14cb51616014d4dba86966f459f16e9.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=30ac7931893e54415df2b3d07a28d96c, UNASSIGN in 179 msec 2023-02-09 16:55:56,555 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/02f3f3de62244b1dbad87ea070d1dbbf.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,556 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/0e2004e6754340fcb512c4d7fb434ab2.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,557 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/55d2bcbb00874f58a2d901119529d79e.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,558 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/64ec367f30854c1cb95f513ad1926d0a.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,559 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/28040a6d692d4e9d9035d409d2419014.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,560 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/7748adcc31ab4d88a0acd2b63745158c.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,561 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a-top, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45->hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668-top] to archive 2023-02-09 16:55:56,562 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 16:55:56,563 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/b841fc2288e7431f8b2e95da5920d4fe.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,564 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/dfe70c05ebd2404985f196f8e25e1660.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,565 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/4101c9f4a5214361a03a913eb094ced7.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,565 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/06656ed91c0e47dfbe6256e36ad731a8.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,566 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/be528a0528584a5a9b3c5c00e7aa6ff5.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,567 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/e2b41d6fe4bd4f6b97caeb019402509a.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,568 DEBUG [StoreCloser-testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/175ca1cb6b9e46fb8eb4eee0de79d668.981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 16:55:56,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:56,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545. 2023-02-09 16:55:56,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 12c9b49df55b6017a6b7a81fa449a545: Waiting for close lock at 1675961756551Running coprocessor pre-close hooks at 1675961756551Disabling compacts and flushes for region at 1675961756551Disabling writes for close at 1675961756551Writing region close event to WAL at 1675961756569 (+18 ms)Running coprocessor post-close hooks at 1675961756571 (+2 ms)Closed at 1675961756572 (+1 ms) 2023-02-09 16:55:56,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,573 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=12c9b49df55b6017a6b7a81fa449a545, regionState=CLOSED 2023-02-09 16:55:56,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675961756573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675961756573"}]},"ts":"1675961756573"} 2023-02-09 16:55:56,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-09 16:55:56,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 12c9b49df55b6017a6b7a81fa449a545, server=jenkins-hbase4.apache.org,43347,1675961647698 in 197 msec 2023-02-09 16:55:56,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-09 16:55:56,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=12c9b49df55b6017a6b7a81fa449a545, UNASSIGN in 200 msec 2023-02-09 16:55:56,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675961756577"}]},"ts":"1675961756577"} 2023-02-09 16:55:56,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-09 16:55:56,580 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-09 16:55:56,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 212 msec 2023-02-09 16:55:56,673 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,673 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 16:55:56,674 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-09 16:55:56,674 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,674 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,675 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-09 16:55:56,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,677 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,677 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,678 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,678 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 16:55:56,681 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,681 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,681 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,683 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/recovered.edits] 2023-02-09 16:55:56,683 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/recovered.edits] 2023-02-09 16:55:56,683 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2, FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/recovered.edits] 2023-02-09 16:55:56,686 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/2bd32efed4b64fc2ad0a96ddf24ad376 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf1/2bd32efed4b64fc2ad0a96ddf24ad376 2023-02-09 16:55:56,686 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b137253ac21b4ece98808c35d4f397ca to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf1/b137253ac21b4ece98808c35d4f397ca 2023-02-09 16:55:56,687 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/02f3f3de62244b1dbad87ea070d1dbbf 2023-02-09 16:55:56,688 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/0e2004e6754340fcb512c4d7fb434ab2 2023-02-09 16:55:56,688 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/1d9832b9a2af48cb98c9f8bf11b9434e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/cf2/1d9832b9a2af48cb98c9f8bf11b9434e 2023-02-09 16:55:56,688 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/0f6312332fd34b0699a4c8d2417949f0 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/cf2/0f6312332fd34b0699a4c8d2417949f0 2023-02-09 16:55:56,689 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/28040a6d692d4e9d9035d409d2419014 2023-02-09 16:55:56,691 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/55d2bcbb00874f58a2d901119529d79e 2023-02-09 16:55:56,691 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c/recovered.edits/90.seqid 2023-02-09 16:55:56,691 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/recovered.edits/90.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545/recovered.edits/90.seqid 2023-02-09 16:55:56,691 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/30ac7931893e54415df2b3d07a28d96c 2023-02-09 16:55:56,692 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/12c9b49df55b6017a6b7a81fa449a545 2023-02-09 16:55:56,692 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/64ec367f30854c1cb95f513ad1926d0a 2023-02-09 16:55:56,693 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/7748adcc31ab4d88a0acd2b63745158c 2023-02-09 16:55:56,694 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf1/b14cb51616014d4dba86966f459f16e9 2023-02-09 16:55:56,696 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/06656ed91c0e47dfbe6256e36ad731a8 2023-02-09 16:55:56,696 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/175ca1cb6b9e46fb8eb4eee0de79d668 2023-02-09 16:55:56,698 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/4101c9f4a5214361a03a913eb094ced7 2023-02-09 16:55:56,699 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/b841fc2288e7431f8b2e95da5920d4fe 2023-02-09 16:55:56,699 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/be528a0528584a5a9b3c5c00e7aa6ff5 2023-02-09 16:55:56,700 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660 to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/dfe70c05ebd2404985f196f8e25e1660 2023-02-09 16:55:56,701 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/cf2/e2b41d6fe4bd4f6b97caeb019402509a 2023-02-09 16:55:56,703 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/recovered.edits/85.seqid to hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/archive/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45/recovered.edits/85.seqid 2023-02-09 16:55:56,704 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/.tmp/data/default/testSplitTableRegion/981cad13cbee8b41b5ca0150bff7cc45 2023-02-09 16:55:56,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 16:55:56,706 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,707 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-09 16:55:56,708 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-09 16:55:56,709 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,709 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-09 16:55:56,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961756709"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:56,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961756709"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:56,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675961756709"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:56,711 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 16:55:56,711 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 981cad13cbee8b41b5ca0150bff7cc45, NAME => 'testSplitTableRegion,,1675961753303.981cad13cbee8b41b5ca0150bff7cc45.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 30ac7931893e54415df2b3d07a28d96c, NAME => 'testSplitTableRegion,,1675961755242.30ac7931893e54415df2b3d07a28d96c.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 12c9b49df55b6017a6b7a81fa449a545, NAME => 'testSplitTableRegion,41,1675961755242.12c9b49df55b6017a6b7a81fa449a545.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 16:55:56,711 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-09 16:55:56,711 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675961756711"}]},"ts":"9223372036854775807"} 2023-02-09 16:55:56,712 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-09 16:55:56,714 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 16:55:56,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 39 msec 2023-02-09 16:55:56,778 INFO [Listener at localhost/39015] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 16:55:56,778 INFO [Listener at localhost/39015] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 16:55:56,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=35255] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 16:55:56,779 INFO [Listener at localhost/39015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-09 16:55:56,793 INFO [Listener at localhost/39015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377 (was 374) - Thread LEAK? -, OpenFileDescriptor=826 (was 823) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=118 (was 111) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=2890 (was 2899) 2023-02-09 16:55:56,793 INFO [Listener at localhost/39015] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 16:55:56,793 INFO [Listener at localhost/39015] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 16:55:56,795 DEBUG [Listener at localhost/39015] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37c223d6 to 127.0.0.1:59211 2023-02-09 16:55:56,795 DEBUG [Listener at localhost/39015] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:56,796 DEBUG [Listener at localhost/39015] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 16:55:56,796 DEBUG [Listener at localhost/39015] util.JVMClusterUtil(257): Found active master hash=257596825, stopped=false 2023-02-09 16:55:56,796 INFO [Listener at localhost/39015] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:55:56,797 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 16:55:56,797 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 16:55:56,797 INFO [Listener at localhost/39015] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 16:55:56,797 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 16:55:56,797 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:55:56,797 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 16:55:56,798 DEBUG [Listener at localhost/39015] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2619c11d to 127.0.0.1:59211 2023-02-09 16:55:56,798 DEBUG [Listener at localhost/39015] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:56,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:55:56,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:55:56,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:55:56,798 INFO [Listener at localhost/39015] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43347,1675961647698' ***** 2023-02-09 16:55:56,798 INFO [Listener at localhost/39015] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 16:55:56,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 16:55:56,799 INFO [Listener at localhost/39015] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37073,1675961647745' ***** 2023-02-09 16:55:56,799 INFO [Listener at localhost/39015] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 16:55:56,799 INFO [RS:0;jenkins-hbase4:43347] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 16:55:56,799 INFO [Listener at localhost/39015] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33383,1675961647775' ***** 2023-02-09 16:55:56,799 INFO [RS:0;jenkins-hbase4:43347] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 16:55:56,799 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 16:55:56,799 INFO [RS:0;jenkins-hbase4:43347] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 16:55:56,799 INFO [RS:1;jenkins-hbase4:37073] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 16:55:56,799 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:56,799 INFO [RS:1;jenkins-hbase4:37073] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 16:55:56,799 DEBUG [RS:0;jenkins-hbase4:43347] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4fa7ac0a to 127.0.0.1:59211 2023-02-09 16:55:56,799 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 16:55:56,799 DEBUG [RS:0;jenkins-hbase4:43347] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:56,799 INFO [Listener at localhost/39015] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 16:55:56,799 INFO [RS:0;jenkins-hbase4:43347] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 16:55:56,799 INFO [RS:1;jenkins-hbase4:37073] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 16:55:56,800 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:56,800 INFO [RS:2;jenkins-hbase4:33383] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 16:55:56,800 INFO [RS:0;jenkins-hbase4:43347] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 16:55:56,800 INFO [RS:2;jenkins-hbase4:33383] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 16:55:56,800 INFO [RS:0;jenkins-hbase4:43347] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 16:55:56,800 INFO [RS:2;jenkins-hbase4:33383] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 16:55:56,800 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 16:55:56,800 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(3304): Received CLOSE for 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:55:56,800 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 16:55:56,800 DEBUG [RS:1;jenkins-hbase4:37073] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11186348 to 127.0.0.1:59211 2023-02-09 16:55:56,800 DEBUG [RS:1;jenkins-hbase4:37073] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:56,800 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37073,1675961647745; all regions closed. 2023-02-09 16:55:56,800 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:56,800 DEBUG [RS:2;jenkins-hbase4:33383] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6afc572b to 127.0.0.1:59211 2023-02-09 16:55:56,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2f94cc9ee2fe56d77bbf16d8818a2c66, disabling compactions & flushes 2023-02-09 16:55:56,800 DEBUG [RS:2;jenkins-hbase4:33383] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:56,800 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 16:55:56,801 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 16:55:56,801 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-09 16:55:56,801 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 16:55:56,801 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-09 16:55:56,801 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1479): Online Regions={2f94cc9ee2fe56d77bbf16d8818a2c66=hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66.} 2023-02-09 16:55:56,801 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 16:55:56,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:55:56,801 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 16:55:56,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:55:56,801 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 16:55:56,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. after waiting 0 ms 2023-02-09 16:55:56,969 DEBUG [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 16:55:56,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:55:56,969 DEBUG [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1505): Waiting on 2f94cc9ee2fe56d77bbf16d8818a2c66 2023-02-09 16:55:56,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2f94cc9ee2fe56d77bbf16d8818a2c66 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-09 16:55:56,968 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 16:55:56,969 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 16:55:56,970 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 16:55:56,970 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-09 16:55:56,986 DEBUG [RS:1;jenkins-hbase4:37073] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs 2023-02-09 16:55:56,986 INFO [RS:1;jenkins-hbase4:37073] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37073%2C1675961647745:(num 1675961649253) 2023-02-09 16:55:56,986 DEBUG [RS:1;jenkins-hbase4:37073] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:56,986 INFO [RS:1;jenkins-hbase4:37073] regionserver.LeaseManager(133): Closed leases 2023-02-09 16:55:56,991 INFO [RS:1;jenkins-hbase4:37073] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-09 16:55:56,992 INFO [RS:1;jenkins-hbase4:37073] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 16:55:56,992 INFO [RS:1;jenkins-hbase4:37073] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 16:55:56,992 INFO [RS:1;jenkins-hbase4:37073] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 16:55:56,992 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 16:55:57,001 INFO [RS:1;jenkins-hbase4:37073] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37073 2023-02-09 16:55:57,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/.tmp/info/85106f3b6b45406e9d635a2de44d2256 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37073,1675961647745 2023-02-09 16:55:57,008 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:55:57,009 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37073,1675961647745] 2023-02-09 16:55:57,009 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37073,1675961647745; numProcessing=1 2023-02-09 16:55:57,009 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/.tmp/info/b3abc55c1bb64243b7e618fe30761000 2023-02-09 16:55:57,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/.tmp/info/85106f3b6b45406e9d635a2de44d2256 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/info/85106f3b6b45406e9d635a2de44d2256 2023-02-09 16:55:57,012 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37073,1675961647745 already deleted, retry=false 2023-02-09 16:55:57,012 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37073,1675961647745 expired; onlineServers=2 2023-02-09 16:55:57,013 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b3abc55c1bb64243b7e618fe30761000 2023-02-09 16:55:57,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/info/85106f3b6b45406e9d635a2de44d2256, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 16:55:57,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2f94cc9ee2fe56d77bbf16d8818a2c66 in 46ms, sequenceid=6, compaction requested=false 2023-02-09 16:55:57,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/namespace/2f94cc9ee2fe56d77bbf16d8818a2c66/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-09 16:55:57,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:57,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:55:57,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2f94cc9ee2fe56d77bbf16d8818a2c66: Waiting for close lock at 1675961756800Running coprocessor pre-close hooks at 1675961756800Disabling compacts and flushes for region at 1675961756800Disabling writes for close at 1675961756969 (+169 ms)Obtaining lock to block concurrent updates at 1675961756969Preparing flush snapshotting stores in 2f94cc9ee2fe56d77bbf16d8818a2c66 at 1675961756969Finished memstore snapshotting hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675961756970 (+1 ms)Flushing stores of hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. at 1675961756971 (+1 ms)Flushing 2f94cc9ee2fe56d77bbf16d8818a2c66/info: creating writer at 1675961756971Flushing 2f94cc9ee2fe56d77bbf16d8818a2c66/info: appending metadata at 1675961756976 (+5 ms)Flushing 2f94cc9ee2fe56d77bbf16d8818a2c66/info: closing flushed file at 1675961756977 (+1 ms)Flushing 2f94cc9ee2fe56d77bbf16d8818a2c66/info: reopening flushed file at 1675961757010 (+33 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2f94cc9ee2fe56d77bbf16d8818a2c66 in 46ms, sequenceid=6, compaction requested=false at 1675961757016 (+6 ms)Writing region close event to WAL at 1675961757017 (+1 ms)Running coprocessor post-close hooks at 1675961757020 (+3 ms)Closed at 1675961757025 (+5 ms) 2023-02-09 16:55:57,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675961649690.2f94cc9ee2fe56d77bbf16d8818a2c66. 2023-02-09 16:55:57,027 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/.tmp/rep_barrier/67cb9083bcd842c0af09c2f0194a1fd0 2023-02-09 16:55:57,030 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 67cb9083bcd842c0af09c2f0194a1fd0 2023-02-09 16:55:57,042 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/.tmp/table/9643014521884066ae5e5d0c68bc0e04 2023-02-09 16:55:57,045 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9643014521884066ae5e5d0c68bc0e04 2023-02-09 16:55:57,046 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/.tmp/info/b3abc55c1bb64243b7e618fe30761000 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/info/b3abc55c1bb64243b7e618fe30761000 2023-02-09 16:55:57,049 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b3abc55c1bb64243b7e618fe30761000 2023-02-09 16:55:57,049 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/info/b3abc55c1bb64243b7e618fe30761000, entries=59, sequenceid=210, filesize=11.1 K 2023-02-09 16:55:57,050 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/.tmp/rep_barrier/67cb9083bcd842c0af09c2f0194a1fd0 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/rep_barrier/67cb9083bcd842c0af09c2f0194a1fd0 2023-02-09 16:55:57,053 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 67cb9083bcd842c0af09c2f0194a1fd0 2023-02-09 16:55:57,053 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/rep_barrier/67cb9083bcd842c0af09c2f0194a1fd0, entries=24, sequenceid=210, filesize=7.5 K 2023-02-09 16:55:57,054 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/.tmp/table/9643014521884066ae5e5d0c68bc0e04 as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/table/9643014521884066ae5e5d0c68bc0e04 2023-02-09 16:55:57,057 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9643014521884066ae5e5d0c68bc0e04 2023-02-09 16:55:57,057 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/table/9643014521884066ae5e5d0c68bc0e04, entries=36, sequenceid=210, filesize=8.0 K 2023-02-09 16:55:57,057 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 87ms, sequenceid=210, compaction requested=false 2023-02-09 16:55:57,061 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-09 16:55:57,062 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 16:55:57,062 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 16:55:57,063 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 16:55:57,063 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675961756801Running coprocessor pre-close hooks at 1675961756801Disabling compacts and flushes for region at 1675961756801Disabling writes for close at 1675961756970 (+169 ms)Obtaining lock to block concurrent updates at 1675961756970Preparing flush snapshotting stores in 1588230740 at 1675961756970Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1675961756970Flushing stores of hbase:meta,,1.1588230740 at 1675961756971 (+1 ms)Flushing 1588230740/info: creating writer at 1675961756971Flushing 1588230740/info: appending metadata at 1675961756980 (+9 ms)Flushing 1588230740/info: closing flushed file at 1675961756980Flushing 1588230740/rep_barrier: creating writer at 1675961757013 (+33 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675961757015 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675961757015Flushing 1588230740/table: creating writer at 1675961757030 (+15 ms)Flushing 1588230740/table: appending metadata at 1675961757032 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675961757032Flushing 1588230740/info: reopening flushed file at 1675961757046 (+14 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675961757050 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675961757054 (+4 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 87ms, sequenceid=210, compaction requested=false at 1675961757057 (+3 ms)Writing region close event to WAL at 1675961757059 (+2 ms)Running coprocessor post-close hooks at 1675961757062 (+3 ms)Closed at 1675961757063 (+1 ms) 2023-02-09 16:55:57,063 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 16:55:57,092 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 16:55:57,092 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 16:55:57,111 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,111 INFO [RS:1;jenkins-hbase4:37073] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37073,1675961647745; zookeeper connection closed. 2023-02-09 16:55:57,112 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@56146a16 rejected from java.util.concurrent.ThreadPoolExecutor@522be40c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,112 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3de974fe] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3de974fe 2023-02-09 16:55:57,112 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:37073-0x1003a01243d0002, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,112 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@53c5a5db rejected from java.util.concurrent.ThreadPoolExecutor@522be40c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,169 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43347,1675961647698; all regions closed. 2023-02-09 16:55:57,170 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33383,1675961647775; all regions closed. 2023-02-09 16:55:57,177 DEBUG [RS:2;jenkins-hbase4:33383] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs 2023-02-09 16:55:57,177 INFO [RS:2;jenkins-hbase4:33383] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33383%2C1675961647775:(num 1675961649253) 2023-02-09 16:55:57,177 DEBUG [RS:2;jenkins-hbase4:33383] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:57,177 INFO [RS:2;jenkins-hbase4:33383] regionserver.LeaseManager(133): Closed leases 2023-02-09 16:55:57,177 DEBUG [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs 2023-02-09 16:55:57,177 INFO [RS:2;jenkins-hbase4:33383] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 16:55:57,177 INFO [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43347%2C1675961647698.meta:.meta(num 1675961649418) 2023-02-09 16:55:57,178 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 16:55:57,178 INFO [RS:2;jenkins-hbase4:33383] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 16:55:57,178 INFO [RS:2;jenkins-hbase4:33383] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 16:55:57,178 INFO [RS:2;jenkins-hbase4:33383] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 16:55:57,179 INFO [RS:2;jenkins-hbase4:33383] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33383 2023-02-09 16:55:57,181 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:57,181 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33383,1675961647775 2023-02-09 16:55:57,181 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@42a60ed1 rejected from java.util.concurrent.ThreadPoolExecutor@4699825d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,182 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:55:57,183 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33383,1675961647775] 2023-02-09 16:55:57,183 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33383,1675961647775; numProcessing=2 2023-02-09 16:55:57,186 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33383,1675961647775 already deleted, retry=false 2023-02-09 16:55:57,186 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33383,1675961647775 expired; onlineServers=1 2023-02-09 16:55:57,186 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-09 16:55:57,186 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-09 16:55:57,187 DEBUG [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/oldWALs 2023-02-09 16:55:57,187 INFO [RS:0;jenkins-hbase4:43347] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43347%2C1675961647698:(num 1675961649253) 2023-02-09 16:55:57,187 DEBUG [RS:0;jenkins-hbase4:43347] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:57,187 INFO [RS:0;jenkins-hbase4:43347] regionserver.LeaseManager(133): Closed leases 2023-02-09 16:55:57,187 INFO [RS:0;jenkins-hbase4:43347] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 16:55:57,187 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 16:55:57,188 INFO [RS:0;jenkins-hbase4:43347] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43347 2023-02-09 16:55:57,189 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43347,1675961647698 2023-02-09 16:55:57,189 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5698ca39 rejected from java.util.concurrent.ThreadPoolExecutor@52a74432[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,189 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 16:55:57,191 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43347,1675961647698] 2023-02-09 16:55:57,191 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43347,1675961647698; numProcessing=3 2023-02-09 16:55:57,192 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43347,1675961647698 already deleted, retry=false 2023-02-09 16:55:57,192 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43347,1675961647698 expired; onlineServers=0 2023-02-09 16:55:57,192 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,35255,1675961646540' ***** 2023-02-09 16:55:57,192 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 16:55:57,193 DEBUG [M:0;jenkins-hbase4:35255] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e6797a2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 16:55:57,193 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,35255,1675961646540 2023-02-09 16:55:57,193 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,35255,1675961646540; all regions closed. 2023-02-09 16:55:57,193 DEBUG [M:0;jenkins-hbase4:35255] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 16:55:57,193 DEBUG [M:0;jenkins-hbase4:35255] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 16:55:57,193 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 16:55:57,193 DEBUG [M:0;jenkins-hbase4:35255] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 16:55:57,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675961648880] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675961648880,5,FailOnTimeoutGroup] 2023-02-09 16:55:57,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675961648878] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675961648878,5,FailOnTimeoutGroup] 2023-02-09 16:55:57,193 INFO [M:0;jenkins-hbase4:35255] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 16:55:57,193 INFO [M:0;jenkins-hbase4:35255] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 16:55:57,193 INFO [M:0;jenkins-hbase4:35255] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 16:55:57,193 DEBUG [M:0;jenkins-hbase4:35255] master.HMaster(1502): Stopping service threads 2023-02-09 16:55:57,193 INFO [M:0;jenkins-hbase4:35255] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 16:55:57,194 ERROR [M:0;jenkins-hbase4:35255] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-09 16:55:57,194 INFO [M:0;jenkins-hbase4:35255] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 16:55:57,194 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 16:55:57,195 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 16:55:57,195 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 16:55:57,195 DEBUG [M:0;jenkins-hbase4:35255] zookeeper.ZKUtil(398): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 16:55:57,195 WARN [M:0;jenkins-hbase4:35255] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 16:55:57,195 INFO [M:0;jenkins-hbase4:35255] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 16:55:57,195 INFO [M:0;jenkins-hbase4:35255] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 16:55:57,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 16:55:57,195 DEBUG [M:0;jenkins-hbase4:35255] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 16:55:57,195 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:55:57,196 DEBUG [M:0;jenkins-hbase4:35255] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:55:57,196 DEBUG [M:0;jenkins-hbase4:35255] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 16:55:57,196 DEBUG [M:0;jenkins-hbase4:35255] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:55:57,196 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.16 KB heapSize=719.07 KB 2023-02-09 16:55:57,209 INFO [M:0;jenkins-hbase4:35255] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.16 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d395d69d6cc4476da453c8ae78fdcc8e 2023-02-09 16:55:57,213 INFO [M:0;jenkins-hbase4:35255] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d395d69d6cc4476da453c8ae78fdcc8e 2023-02-09 16:55:57,213 DEBUG [M:0;jenkins-hbase4:35255] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d395d69d6cc4476da453c8ae78fdcc8e as hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d395d69d6cc4476da453c8ae78fdcc8e 2023-02-09 16:55:57,217 INFO [M:0;jenkins-hbase4:35255] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d395d69d6cc4476da453c8ae78fdcc8e 2023-02-09 16:55:57,217 INFO [M:0;jenkins-hbase4:35255] regionserver.HStore(1080): Added hdfs://localhost:44699/user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d395d69d6cc4476da453c8ae78fdcc8e, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-09 16:55:57,217 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegion(2947): Finished flush of dataSize ~608.16 KB/622752, heapSize ~719.05 KB/736312, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=1282, compaction requested=false 2023-02-09 16:55:57,218 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 16:55:57,218 DEBUG [M:0;jenkins-hbase4:35255] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675961757195Disabling compacts and flushes for region at 1675961757195Disabling writes for close at 1675961757196 (+1 ms)Obtaining lock to block concurrent updates at 1675961757196Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675961757196Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622752, getHeapSize=736312, getOffHeapSize=0, getCellsCount=1499 at 1675961757196Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675961757197 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675961757197Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675961757201 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675961757201Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675961757214 (+13 ms)Finished flush of dataSize ~608.16 KB/622752, heapSize ~719.05 KB/736312, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=1282, compaction requested=false at 1675961757218 (+4 ms)Writing region close event to WAL at 1675961757218Closed at 1675961757218 2023-02-09 16:55:57,221 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/ce79d20f-d72b-f789-c449-e8a7e1dad1a9/MasterData/WALs/jenkins-hbase4.apache.org,35255,1675961646540/jenkins-hbase4.apache.org%2C35255%2C1675961646540.1675961648243 not finished, retry = 0 2023-02-09 16:55:57,285 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,285 INFO [RS:2;jenkins-hbase4:33383] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33383,1675961647775; zookeeper connection closed. 2023-02-09 16:55:57,285 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4070ad35 rejected from java.util.concurrent.ThreadPoolExecutor@4699825d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,285 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:33383-0x1003a01243d0003, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,285 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@92f6e85] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@92f6e85 2023-02-09 16:55:57,285 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@38d31401 rejected from java.util.concurrent.ThreadPoolExecutor@4699825d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,322 INFO [M:0;jenkins-hbase4:35255] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 16:55:57,322 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 16:55:57,323 INFO [M:0;jenkins-hbase4:35255] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:35255 2023-02-09 16:55:57,325 DEBUG [M:0;jenkins-hbase4:35255] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,35255,1675961646540 already deleted, retry=false 2023-02-09 16:55:57,385 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,385 INFO [RS:0;jenkins-hbase4:43347] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43347,1675961647698; zookeeper connection closed. 2023-02-09 16:55:57,385 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@36a1d88b rejected from java.util.concurrent.ThreadPoolExecutor@52a74432[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,385 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): regionserver:43347-0x1003a01243d0001, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,385 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4e16535d rejected from java.util.concurrent.ThreadPoolExecutor@52a74432[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,385 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@34cf69c8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@34cf69c8 2023-02-09 16:55:57,386 INFO [Listener at localhost/39015] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-09 16:55:57,485 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,485 INFO [M:0;jenkins-hbase4:35255] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,35255,1675961646540; zookeeper connection closed. 2023-02-09 16:55:57,485 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7b1ed45f rejected from java.util.concurrent.ThreadPoolExecutor@17eb6abd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,486 DEBUG [Listener at localhost/39015-EventThread] zookeeper.ZKWatcher(600): master:35255-0x1003a01243d0000, quorum=127.0.0.1:59211, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 16:55:57,486 ERROR [Listener at localhost/39015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d2cd8f7 rejected from java.util.concurrent.ThreadPoolExecutor@17eb6abd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 16:55:57,487 WARN [Listener at localhost/39015] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 16:55:57,490 INFO [Listener at localhost/39015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 16:55:57,578 WARN [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid 762ffd31-8114-414e-bb4b-20a4d577f100) service to localhost/127.0.0.1:44699 2023-02-09 16:55:57,581 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data5/current/BP-1895155465-172.31.14.131-1675961642820] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 16:55:57,581 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data6/current/BP-1895155465-172.31.14.131-1675961642820] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 16:55:57,600 WARN [Listener at localhost/39015] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 16:55:57,607 INFO [Listener at localhost/39015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 16:55:57,695 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 16:55:57,735 WARN [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 16:55:57,735 WARN [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid e1d56305-25f8-42c9-ab95-be1e251b8441) service to localhost/127.0.0.1:44699 2023-02-09 16:55:57,736 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data3/current/BP-1895155465-172.31.14.131-1675961642820] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 16:55:57,736 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data4/current/BP-1895155465-172.31.14.131-1675961642820] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 16:55:57,738 WARN [Listener at localhost/39015] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 16:55:57,741 INFO [Listener at localhost/39015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 16:55:57,843 WARN [BP-1895155465-172.31.14.131-1675961642820 heartbeating to localhost/127.0.0.1:44699] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1895155465-172.31.14.131-1675961642820 (Datanode Uuid 627681a5-99ed-41c7-b973-a8e3d411bd34) service to localhost/127.0.0.1:44699 2023-02-09 16:55:57,844 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data1/current/BP-1895155465-172.31.14.131-1675961642820] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 16:55:57,844 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/43dea4dc-c04b-aeb4-3203-8d4ae4fdb6d4/cluster_6fd0ba51-d1e9-8b64-9fa9-c442722b79b6/dfs/data/data2/current/BP-1895155465-172.31.14.131-1675961642820] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 16:55:57,864 INFO [Listener at localhost/39015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 16:55:58,055 INFO [Listener at localhost/39015] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 16:55:58,282 INFO [Listener at localhost/39015] hbase.HBaseTestingUtility(1293): Minicluster is down